Lesson 5 · Phase 4: Agent Systems Engineering · visual edition

Agent Systems Engineering — how production AI executes work

Q: Define agent systems engineering and name its core concerns.

Hit these points: agent engineering = make one agent work (prompt + tools + loop) → agent systems engineering = run it as a production system → core concerns: planning, state management, execution engine, loop control, human-in-the-loop, failure recovery, orchestration, long-running durability, cost, observability → the through-line: an agent run is a long-lived, non-deterministic distributed computation, and all of those follow from that.

Q: Walk through the agent loop and say exactly when it stops.

Hit these points: observe (gather results/context) → think (model decides next action) → act (tool call) → repeat → completion stops: model emits a final answer with no tool call, or calls an explicit done/submit tool, ideally confirmed by an external verifier → safety stops (runtime): step cap, wall-clock timeout, token/cost budget, repetition detector, human interrupt → the model proposes stopping; the runtime decides — stops are enforced outside the prompt.

Q: What is agent state, and why is the LLM called stateless?

Hit these points: state = goal/current task + the plan + message history + tool outputs/intermediate results → the LLM is a pure function f(input)→output with no memory between calls → the runtime makes it stateful by re-assembling and re-sending context every turn → the context window is the slice of state the model sees this call → externalize state and you can scale, checkpoint, and recover; trap it in process memory and you can't.

Q: Name the four rungs of agent failure recovery and a one-line example of each.

Hit these points: retry — transient idempotent tool error, backoff + jitter, bounded → fallback — degrade to a smaller model / cache / alternate tool → compensate — irreversible side effect already happened, run a registered undo (Saga) → escalate — recovery exhausted, dead-letter + human → climb only as far as needed, and wrap tools in circuit breakers.

Q: Why is success detection hard, and how do you make stopping reliable?

Hit these points: the loop's natural stop is the model self-reporting "done," and the model is frequently wrong — it quits early on unfinished work or loops when finished → so never trust self-report alone: add an external verifier (tests pass, schema validates, artifact exists) → and add runtime safety stops independent of the model (step cap, wall-clock timeout, budget, repetition detector, human halt) → the senior framing: completion is a verification problem, termination is a control problem — solve them separately, both in the runtime.

Q: Why is multi-agent so often the wrong call, and how do you decide?

Hit these points: teams reach for it for prestige or to "add intelligence," but intelligence is a model/context lever, not a topology lever → each split buys coordination overhead, a context handoff that can drop information, harder debugging ("which agent broke?"), and N×M cost → decide like a microservice split: only when work is genuinely independent & parallel, context is separable and exceeds a window, or subtasks need distinct toolsets → default to a single agent (or a router) and earn each split with a real boundary.

Q: Map agent failures onto distributed-systems patterns — and name the one agents get wrong.

Hit these points: tool failure = downstream service failure → retry + circuit breaker; model failure = bad RPC response → validate + repair + retry on another tier; context failure = cache pollution → compress/re-rank/re-inject; planning failure = bad query plan → re-plan; execution failure = partial write → saga / idempotency → the one agents get wrong is compensation: there's no rollback for a sent email or an issued refund, so every irreversible action needs a registered compensating action — plus idempotency keys upstream so retries don't double-apply.

Q: An agent's bill 10×'d overnight. Diagnose and design the controls.

Hit these points: first suspects — a runaway loop (no progress, repeated calls), context bloat (history re-sent and growing), or a multi-agent fan-out amplifying both → cost is super-linear because input grows each step, so the unit to watch is cost per completed task, attributed per tenant → controls: prompt caching on the stable prefix, history compaction, model tiering, per-task step + token budgets with a kill switch, and an anomaly alert at ~3× baseline so it trips an incident before the invoice → close the loop with the runtime guards from loop control so "runaway" can't happen silently.

Q: Design the execution + durability layer for an agent that runs for hours and survives crashes.

Hit these points: model the run as a durable workflow, not an in-memory loop — the agent loop is the non-deterministic step inside deterministic orchestration → state (work-list with per-item status, plan, cursor, artifacts) lives in a durable store; checkpoint after each meaningful item → on restart, resume from the last checkpoint via deterministic replay, skipping completed items → every replayed/ retried step is idempotent (keys) so resume can't double-charge or duplicate → layer recovery (retry→fallback→compensate→escalate), HITL gates before irreversible writes, per-item budgets, and per-item traces + an immutable audit log → name the through-line: borrow durable-execution discipline (Temporal-style) so a non-deterministic worker gets deterministic retries, checkpoints, and observability.

Q: An engineer wants to convert a stable, well-defined business process into an agent. Make the call.

Hit these points: if the steps are known and rarely change, an agent is the wrong tool — you'd pay non-determinism, latency, and token cost for control flow a workflow or plain code handles deterministically, debuggably, and testably → "agent for everything" is the dominant anti-pattern → the senior move: put it on the workflow↔agent spectrum — fixed path → workflow; open-ended path the model must decide → agent; hybrid → workflow with agent steps only where the path is genuinely unknown → reserve agency for the irreducible uncertainty and keep the rest deterministic → this is also cheaper to operate and far easier to get past a review.

~45 min · 14 modules + deliverables · planning · state · execution · loop control · failure recovery · cost · observability

Published Jun 21, 2026

Agent Systems EngineeringPlanningExecution EngineLoop ControlFailure Recovery

Your bar: explain how a production agent actually executes work end-to-end — plan, hold state, run the loop, recover from failure, and survive for hours — and run an architecture review that separates real engineering from autonomy theatre. Every module maps the agent onto something you already trust: a runtime, a workflow engine, a distributed system.

An agent run = a long-lived, non-deterministic distributed computation

the runtime is the execution engine — not the model

every failure mode has a distributed-systems precedent

autonomy is a dial set by blast radius

What Is Agent Systems Engineering?

Agent engineering makes one agent work. Agent systems engineering makes it execute work reliably, for hours, under failure, at a cost you can defend. Same jump as script → service.

Each step right adds a class of problem that did not exist before. ASE is everything in the third box — the part that is engineering, not prompting.

Is The discipline of running agents as production systems: planning, state, an execution engine, loop control, recovery, orchestration, durability, cost, and observability — the operational layer around the model.

Why it exists A demo agent is a happy-path script. Production adds crashes, timeouts, hostile inputs, runaway loops, multi-hour tasks, and a bill. None of that is fixed by a better prompt; it is fixed by systems engineering.

Like (world) A line cook who can make one dish vs running a restaurant at dinner rush: inventory, tickets, timing, substitutions when the salmon runs out, and a P&L. The recipe was never the hard part.

Like (code) A main() that calls an API once vs a service: retries, queues, idempotency, observability, autoscaling, on-call. ASE is the move from “it ran on my machine” to “it runs for everyone, always.”

✗ “We have a working agent, so we have an agent system.”

✓ A working agent is the LLM-plus-loop core. The system is the 80% around it: what happens on step 40 of a 60-step task when a tool times out and the user has gone home.

🧮 Memory rule: An agent is a function; an agent system is a distributed, long-lived, non-deterministic computation. Every module in this lesson is one consequence of that sentence.

Memory check — 3 questions

In one line, the difference between agent engineering and agent systems engineering? → Engineering = make one agent work (prompt + tools + loop). Systems engineering = make it execute reliably in production (state, recovery, durability, cost, observability).
Name three problems that appear only in the third box (agent system) and not the second (agent). → Any three of: re-planning, orchestration across agents, crash recovery / durability, cost control at scale, observability/audit, long-running persistence.
Why can’t a better model close that gap? → The gap is operational, not cognitive — crashes, budgets, hostile inputs, and multi-hour durability are runtime concerns the model never sees.

Planning Systems — goal → plan → execution

A goal is not executable. Planning turns it into ordered, checkable steps — and re-planning is what separates a system that adapts from one that confidently marches off a cliff.

The plan is a hypothesis, not a contract. Execution feeds reality back; the planner revises the unstarted steps. A plan with no re-plan path is a brittle script.

DecompositionBreak a goal into steps small enough to execute and check. Too coarse = unverifiable; too fine = token waste.

HierarchicalPlan-of-plans: high-level phases, each expanded just-in-time. Mirrors epic → story → task.

DynamicPlan one step ahead and re-decide (ReAct), instead of committing the whole plan up front.

Re-planningOn failure or new info, discard the stale tail of the plan and regenerate it. The adaptive core.

Plan-then-execute Generate the full plan first, then run it (Plan-and-Solve / Plan-and-Execute²). Cheaper, auditable, parallelizable — but rigid if the world shifts mid-run.

Interleaved (ReAct) Think one step, act, observe, repeat (ReAct¹). Maximally adaptive — but more LLM calls, and it can wander without a plan to anchor it.

Like (world) A surgeon’s plan vs the operation. There is a plan, but the moment they open you up and find something unexpected, they re-plan. Sticking to a wrong plan is how people die.

Like (code) A query planner. The optimizer builds a plan from statistics, but an adaptive executor can re-plan mid-flight when row estimates turn out wrong. Static plan = prepared statement; dynamic = adaptive execution.

Task	Decomposition	Re-plan trigger
Fix an authentication bug	Reproduce → locate → patch → test → verify	Repro fails, or the “fix” breaks another test
Investigate a production outage	Gather signals (logs/metrics/traces in parallel) → hypothesize → confirm → mitigate	Hypothesis disproven by evidence → new branch
Generate a monthly billing report	Pull usage → rate → aggregate → render → reconcile	Mostly static — this is a workflow, not an agent (see M14)

✗ “A good agent makes a perfect plan up front.”

✓ A good agent makes a cheap, revisable plan and updates it as evidence arrives. Plan quality matters less than re-plan discipline.

🧮 Memory rule: The plan is a hypothesis; re-planning is the algorithm. If the architecture can’t replace the stale tail of a plan, it can’t handle the real world.

ReAct vs Plan-and-Execute — when do you pick each?

Pick Plan-and-Execute when the task is decomposable up front and steps can run in parallel or be audited/approved before execution (cheaper, fewer LLM calls, you can show the plan to a human). Pick ReAct (interleaved) when each step’s result genuinely changes what to do next — debugging, investigation, open-ended research. In production many systems are hybrid: a coarse plan up front, ReAct within each phase, and a re-plan checkpoint between phases.¹

Memory check — 3 questions

What are the three levels of decomposition granularity to balance? → Too coarse (unverifiable), right-sized (executable + checkable), too fine (token waste / overhead).
What is the single most important property of a planning system? → The ability to re-plan — discard the stale tail and regenerate when reality diverges from the plan.
Why is the billing report arguably not an agent task? → The steps are known and stable; a deterministic workflow is cheaper, testable, and debuggable. Agency buys nothing.

State Management — what the agent “knows” right now

The LLM is stateless: pure f(input)→output. Every appearance of memory is the runtime threading state back in. Get this wrong and you get amnesia, or a context that poisons itself.

State lives in the runtime; the context window is just the slice of it the model sees this turn. "Memory" is a retrieval-and-assembly job, not a property of the model.

Stateless agent

Each request is independent; no carry-over.
Trivially horizontally scalable; any worker handles any request.
Crash-safe by default — nothing to lose.
Cost: re-establish context every time; can’t do multi-step tasks that span requests.

Stateful agent

Carries plan + history across steps; can do real multi-step work.
Needs a state store, session affinity or externalized state, and a recovery story.
Crash mid-task = lost work unless checkpointed (→ M9).
Failure mode: context bloat & poisoning — stale/wrong state degrades every later step.

Like (world) A consultant with amnesia who reads the entire case file before every meeting. The “file” is your state store; what they can hold in their head for one meeting is the context window.

Like (code) HTTP is stateless; the session is reconstructed from a cookie + server-side store each request. The LLM is the stateless handler; the runtime is the session layer.

✗ “The model remembers the conversation.”

✓ The model sees only what is in this call’s context. The runtime re-sends history every turn; remove it and the “memory” vanishes instantly.

🧮 Memory rule: The model is stateless; the runtime makes it stateful by re-assembling context every turn. Externalize state and you can scale, checkpoint, and recover. Trap it in process memory and you can’t.

Memory check — 3 questions

Where does agent state actually live, and what is the context window relative to it? → State lives in the runtime/store; the context window is the slice of state assembled for one call.
One operational advantage of stateless and one of stateful? → Stateless: trivial scaling + crash-safety. Stateful: real multi-step tasks (at the cost of a store + recovery).
What is context poisoning? → Stale/wrong/irrelevant state accumulating in the window and degrading every subsequent step’s output.

Execution Engines — how steps actually run

Goal → task → subtask → execution. The engine decides the shape: sequential, parallel, or branching. The twist vs a workflow engine: the agent draws the graph at runtime, not at deploy time.

Sequential when steps depend on each other; parallel to cut wall-clock on independent work; branching to choose a path from results. A workflow engine fixes this graph in code; an agent generates it as it goes.

System	Who defines the graph	Determinism	Best at
Workflow engine (Airflow, Temporal)	Developer, at deploy time	Deterministic	Known multi-step pipelines
Queue system (SQS, Kafka)	Developer; decoupled producers/consumers	At-least-once delivery	Buffering, backpressure, fan-out
Distributed job system (k8s Jobs, Spark)	Developer; scheduler places work	Deterministic placement	Throughput over a fixed DAG
Agent execution engine	The model, at runtime	Non-deterministic	Open-ended tasks with an unknown path

Is The component that turns the plan into running steps — ordering them, fanning out independent work, gathering results, and choosing branches — while enforcing the runtime’s limits on each step.

Why it exists Sequential-only execution wastes wall-clock on independent work; un-checked branching wanders. The engine is where parallelism, ordering, and dependency tracking are made explicit and safe.

Like (world) A kitchen expediter: some dishes must follow an order (sear before plating), some run in parallel (three pans), and some depend on a taste-test result (more salt or not).

Like (code) A DAG executor — but the DAG is emitted by a non-deterministic planner step by step, so you need idempotency and tracing that a static Airflow DAG gets for free.

✗ “Agents and workflow engines are competitors.”

✓ They compose. Production systems run agents inside a durable workflow engine so the non-deterministic part gets deterministic retries, checkpoints, and observability (→ M9).

🧮 Memory rule: A workflow fixes the graph at deploy time; an agent draws it at runtime. That single difference is the source of agents’ power and all of their operational pain.

Memory check — 3 questions

The three execution shapes and when to use each? → Sequential (dependent steps), parallel (independent work, cut wall-clock), branching (choose path from a result).
The defining difference between a workflow engine and an agent execution engine? → Who defines the graph and when: developer-at-deploy (deterministic) vs model-at-runtime (non-deterministic).
Why run an agent inside a workflow engine? → To borrow durable retries, checkpointing, and observability for the non-deterministic loop.

Loop Control — observe → think → act → repeat

The loop is trivial to write and brutal to control. The hard questions are all about stopping: when, how do you know it worked, and how do you guarantee it can’t run forever or burn the budget.

The model proposes continuing or stopping; the runtime decides. Green = legitimate completion (and verify it). Red = safety stops that must exist independently of the model's judgment.

When does it stop?No tool call / explicit done tool — or a runtime limit trips. Self-report is necessary but never sufficient.

How know success?Don’t trust the model. Run an external verifier: tests pass, schema validates, the artifact exists.

Prevent infinite loopStep cap + wall-clock timeout + repetition detector + budget. Defense in depth; any one can fail.

RetriesBounded, backoff + jitter, idempotent. On retry, change strategy — don’t repeat the identical failing call.

Like (world) A search party doesn’t stop when a searcher says “found them” — it confirms. And it has a hard rule: turn back at sunset regardless. Self-report plus an independent stop.

Like (code) A retry library + a watchdog timer + a circuit breaker around a while(!done). The loop body is one line; the control plane around it is the engineering.

✗ “The agent knows when it’s done.”

✓ The model guesses it’s done and is regularly wrong — declaring victory early or looping when finished. Success needs an external check; safety needs runtime stops.

🧮 Memory rule: The model proposes stopping; the runtime decides. Success = external verifier. Safety = step cap + timeout + repetition + budget. Both, always.

Your agent occasionally loops forever and runs up cost. Where do you fix it — and why is it not “the model”?

Fix it in the runtime, because looping is a control-plane failure, not a reasoning failure. Add: a hard step cap and a wall-clock timeout (independent, since slow legitimate steps exist); a repetition detector (same tool + same args N times → halt); a token/cost budget with a kill switch and an anomaly alert at ~3× baseline. Then design out the two root causes: context saturation (tool output crowds out the goal → compress / re-inject the goal) and tool-error spirals (retrying an identical failing call → bounded retries + a different strategy on retry). Prompt tweaks can reduce the odds but can’t guarantee termination — only the runtime can.

Memory check — 3 questions

What are the two categories of stop condition? → Completion (final answer / done tool / verifier passes) and safety (step cap, timeout, repetition, budget, human halt).
Why is model self-report insufficient for success detection? → The model is frequently wrong about being done — external verification (tests/schema/artifact) is the only reliable signal.
On retry, what must change and what must be guaranteed? → Change the strategy (don’t repeat the identical failing call); guarantee idempotency so the retry can’t double-apply a side effect.

Human In The Loop — autonomy is a dial, not a switch

The question is never “autonomous or not.” It’s “which actions, at what blast radius, behind which gate.” Reads earn autonomy fast; irreversible writes earn it slowly.

One agent, four gates — chosen per action by reversibility and blast radius. "Delete workspace" and "read logs" should never sit behind the same policy.

Pre-approval (interrupt & wait)Agent proposes a write, pauses, a human approves/edits/rejects, then it proceeds. Needs durable state so the pause can outlive the process (→ M9).

Post-hoc reviewAgent acts on low-risk items autonomously; a human samples/audits after. Cheap, but only for reversible actions.

Confidence-gatedAuto below a confidence/threshold; escalate above it. The cutoff must be tuned on real data, not guessed.

Escalation / dead-letterOutside scope or low confidence → route to a human queue rather than hallucinate an answer or force an action.

✗ “Full autonomy is the goal; HITL is a crutch we’ll remove.”

✓ HITL is a permanent design tool. Even mature systems keep gates on irreversible, high-blast-radius actions — the gate placement just gets sharper over time.

🧮 Memory rule: Set autonomy per action by reversibility × blast radius. Reversible reads run free; irreversible mass actions sit behind a gate forever. Start supervised, loosen on evidence.

Memory check — 3 questions

What two properties of an action set its autonomy level? → Reversibility and blast radius (how much damage the worst single call does).
What does a pre-approval gate require from the runtime? → Durable, resumable state so the agent can pause for a human and survive a restart while waiting.
What should an agent do when a request is outside its scope or below confidence? → Escalate to a human queue (dead-letter), not hallucinate an answer or force a risky action.

Failure Recovery — the agent is a distributed system

Five things fail: the tool, the model, the context, the plan, the execution. Every one has a distributed-systems precedent. You don’t need new theory — you need to apply the old theory.

Recovery is an escalation ladder: cheap-and-local first, expensive-and-human last. The one agents get wrong is compensation — you can't roll back a sent email, so you need a defined undo.

Failure type	Looks like	Primary recovery	Distributed-systems twin
Tool	Timeout, 5xx, rate limit, bad output	Retry (idempotent) → fallback tool	Downstream service failure + circuit breaker
Model	Refusal, malformed/invalid tool call, hallucinated args, timeout	Re-prompt / repair, schema-validate, retry on a different tier	Bad RPC response → validate + retry
Context	Overflow, lost-in-the-middle, poisoned by stale data	Compress, re-rank, re-inject goal, prune	Cache pollution / memory pressure
Planning	Wrong decomposition, impossible step, wrong order	Re-plan (M2); detect via no-progress check	Bad query plan → re-plan
Execution	Right plan, wrong action; partial completion; double side effect	Idempotency keys, compensation, checkpoint + resume	Partial write → saga / 2-phase commit

Like (world) Aviation: a checklist for each failure, a backup system to fall back to, and — when neither works — declare an emergency and hand to ATC. Pilots don’t improvise recovery; they escalate a ladder.

Like (code) Resilience4j / Polly around every external call: retry + circuit breaker + bulkhead + timeout + fallback. The agent’s tools are exactly those external calls.

✗ “Retry handles failures.”

✓ Retry only handles transient, idempotent failures. Retrying a non-idempotent write double-charges; retrying a refusal loops. You need the whole ladder, matched to the failure type.

🧮 Memory rule: Retry → fallback → compensate → escalate. The agent-specific trap is compensation: side effects with no rollback need a defined undo, or you can’t recover safely.

An agent sent a wrong email / issued a wrong refund mid-task, then the task failed. How do you recover?

You can’t roll it back — the side effect escaped the system — so this is the Saga pattern: every irreversible action needs a registered compensating action. Wrong email → send a correction/retraction; wrong refund → reverse the charge (itself idempotent, keyed). The runtime should record each completed side effect in the durable log so that on failure it can run the compensations for exactly the steps that succeeded — no more, no less. Prevention sits upstream: idempotency keys so a retry can’t double-issue, and HITL gates (M6) on the irreversible actions so the worst ones never auto-fire in the first place.

Memory check — 3 questions

The four rungs of the recovery ladder, in order? → Retry → fallback → compensate → escalate.
When is retry the wrong tool? → Non-idempotent side effects (double-charge) and non-transient failures (refusals, bad plans) — it loops or duplicates.
What pattern recovers an irreversible side effect? → Compensation (Saga) — a registered undo action, since there’s no rollback for “already sent”.

Orchestration — one agent, a router, or a coordinator + workers

Three topologies, increasing cost. Most teams jump to coordinator+workers because it looks sophisticated. Justify every split exactly like you’d justify carving out a microservice.

Single agentOne loop, one context, one toolset. The default. Easiest to build, debug, and reason about. Start here, always.

RouterOne agent classifies intent and dispatches to a specialized handler. Cheap; great when inputs are diverse but each path is simple.

Coordinator + workersA lead decomposes, delegates to isolated workers, synthesizes. For genuinely parallel or separable-context work.

Over-engineeredAgents talking to agents for sequential, shared-state work. Pays coordination cost for nothing. The common failure.

Delegation The coordinator owns the goal; it hands a worker one scoped subtask plus exactly the context that worker needs. There is no shared memory — under-specify and the worker fails blind.

Routing Classify, then dispatch. The router owns the decision, not the work. Failure mode: misroute → the wrong specialist confidently handles the wrong thing.

Ownership Exactly one component owns each task’s outcome and its retries. Diffuse ownership = “who was supposed to handle the failure?” = dropped work.

Like (code) Monolith (single) → API gateway with routing (router) → orchestrator fronting microservices (coordinator+workers). The trade-offs transfer one-for-one.

Use multi-agent when…	Stay single-agent when…
Subtasks are genuinely independent & parallel (fan out over 20 files)	Work is sequential with shared state
Context is separable and exceeds one window	It all fits in one context
Subtasks need distinct toolsets / permissions	One toolset covers it
You can attribute & trace each agent independently	You’d struggle to debug “which agent broke”

✗ “More agents = more intelligence.”

✓ More agents = more coordination overhead, more handoffs that drop context, harder debugging, and N×M cost. Intelligence is a model/context lever, not a topology lever.

🧮 Memory rule: Justify a new agent like a new microservice: only when a real boundary demands it. Default to the monolith; earn every split with independent, parallel, or separable-context work.

Memory check — 3 questions

The three legitimate topologies in increasing cost? → Single agent → router → coordinator + workers.
What must the coordinator do that shared-memory systems get for free? → Explicitly assemble and pass each worker’s context, then synthesize outputs — there is no shared state.
The one-line test for “is multi-agent justified”? → Is the work genuinely independent/parallel or separable-context? If it’s sequential with shared state, stay single-agent.

Long-Running Tasks — surviving minutes, hours, days

A 6-hour task will meet a deploy, a crash, or a rate-limit pause. The in-memory while loop dies with the process. The fix is the durable-execution model: persist, checkpoint, resume.

Checkpoint after meaningful steps; on crash, resume from the last good checkpoint. For that to be safe, replayed steps must be idempotent — otherwise resume re-charges, re-emails, re-creates.

Horizon	Example	What must persist	Dominant risk
Minutes	Deep research, multi-file refactor	Plan + partial results in memory/store	Rate-limit pauses, context overflow
Hours	Large code migration, dataset processing	Checkpointed state to durable store	Process restart / deploy mid-run
Days	Customer investigation, monitoring loop	Durable workflow + event log; resumable across hosts	Drift, stale context, cost creep, human handoff

Checkpointing Snapshot enough state (history, plan, cursor, artifacts) to resume from a known-good point — not from zero. Frequency trades durability against overhead.

Durable execution Model the run as a workflow whose state is an event log; on restart, deterministically replay to rebuild state, then continue (the Temporal model⁴).

Like (world) A long expedition with base camps. You don’t restart from sea level after a storm — you resume from the last camp. Camps are checkpoints.

Like (code) Spark stage checkpoints / a resumable DB migration with a version cursor / Temporal workflow replay. The agent loop is just the non-deterministic step inside it.

✗ “We’ll just re-run it if it dies.”

✓ Re-running a stateful, side-effecting task from zero repeats every completed write (duplicate charges, emails, PRs). Resume-from-checkpoint + idempotency is the only safe model.

🧮 Memory rule: Long-running = durable workflow, not an in-memory loop. Persist → checkpoint → resume (replay), and make every replayed step idempotent.

Design a multi-hour code-migration agent that must survive deploys and crashes. What’s the skeleton?

Run it as a durable workflow, not a process-bound loop. State: the work-list (files/modules to migrate) with a per-item status, the plan, and a cursor — all in a durable store, not memory. Granularity: one item = one checkpoint; complete an item, mark it done atomically, then move on. Idempotency: each transform keyed so a replay of an already-done item is a no-op (guards against resume re-applying). Recovery: on restart, load state, skip done items, resume the in-flight one. Control: step/cost budget per item and overall; HITL gate before anything irreversible lands (e.g. opening PRs vs pushing to main). Observability: per-item trace + an audit log so you can answer “what did it change at 2am three days ago” without re-running it. The agent loop is the non-deterministic inside of each item; the durability lives in the workflow around it (→ M4).

Memory check — 3 questions

Why can’t an hours-long task be a plain in-memory loop? → It dies with the process (deploy/crash/restart) and loses all in-flight state.
What makes resume-from-checkpoint safe? → Idempotent replayed steps — so re-running completed steps doesn’t duplicate side effects.
What is the durable-execution model in one line? → Persist state as an event log; on restart, deterministically replay to rebuild state, then continue.

Cost Engineering — why agents cost super-linearly

A demo costs cents. The same agent at scale, or in a loop, costs thousands — because history is re-sent every step, so cost compounds. The unit to manage is cost per completed task.

Each step re-sends the growing prefix, so a 30-step task isn't 30× one call — it's the sum of a growing context. Prompt caching and compaction flatten the curve; multi-agent steepens it.

Lever	What it does	Typical effect
Prompt caching	Reuse the stable system/context prefix across calls	Large cut on re-sent input tokens
History compaction	Summarize / prune old turns so context stops growing	Caps the super-linear term
Model tiering	Cheap model for routing/navigation; strong model for the hard step	Most steps at a fraction of the cost
Step / cost budget	Hard cap per task + kill switch	Bounds the worst case
Cost attribution	Per task + per tenant, with anomaly alert (~3× baseline)	Runaway trips an incident before the invoice

Like (world) A taxi meter that re-charges for the whole trip so far at every block. The only way to control the fare is fewer blocks (steps) and a cheaper rate (tier) — and a hard ceiling.

Like (code) Egress + per-request compute that scales with payload size, where the payload grows each call. You cache the static prefix and trim the payload, exactly as you would a chatty N+1 API.

✗ “Token cost is linear in the number of steps.”

✓ It’s super-linear: input grows each step because history is re-sent. Cost per task — not per call — is the number that bites, and multi-agent multiplies it.

🧮 Memory rule: Manage cost per completed task, not per call. Cache the prefix, compact history, tier the model, budget + alert. A runaway loop is a cost incident, not just a bug.

Memory check — 3 questions

Why is cost super-linear in steps? → The whole conversation history is re-sent as input every step, so input tokens grow each call.
The two levers that flatten the curve? → Prompt caching (stable prefix) and history compaction; model tiering also cuts per-step cost.
What’s the right unit of cost to track and govern? → Cost per completed task, attributed per tenant, with an anomaly alert and a kill switch.

Observability — “why did the agent do that?”

A non-deterministic system you can’t replay is unknowable. The answer to “why” is always the same: show the exact context the model saw at that step. If you didn’t capture it, you can’t answer.

One trace per run, one span per step, child spans per LLM/tool call. The span that captures the assembled context is what turns "why did it do that?" from a guess into a replay.⁵

TracingA span per step/call with input, output, tokens, cost, latency, status — keyed by trace + agent + session id.

Run historyThe ordered step list: plan, decisions, tool calls, results. The narrative of what happened.

Reasoning historyThe model’s intermediate thoughts. Useful for debugging, sensitive for logging — redact.

Audit trailImmutable, append-only record of every mutation: who/what/when, for compliance & post-mortems.

Like (world) An aircraft flight recorder. After an incident you don’t ask the pilot to remember — you replay the black box. The trace is the agent’s black box.

Like (code) Distributed tracing (OpenTelemetry) for a request that fans out across services — plus the one agent-specific field: the assembled prompt/context each step saw.

✗ “We log the inputs and outputs, so we’re observable.”

✓ Without the assembled context per step you can’t reproduce a decision — the model acted on what was in the window, which is built from retrieval + history, not just the user input.

🧮 Memory rule: If you can’t replay the exact context a step saw, you can’t explain the decision. Trace the assembled input, not just the I/O. Audit log every mutation, immutably.

A user says the agent did something wrong three days ago. How do you investigate without re-running it?

Pull the trace by id for that run and walk the step spans. For the offending step, read the assembled context it received — that tells you whether the cause was a retrieval failure (wrong/missing context), a reasoning failure (right context, wrong decision), or a tool failure (right decision, tool errored). Cross-check the audit log for the mutation it performed. This is why the context must be captured at write time: three days later you cannot reconstruct what retrieval returned or what the history looked like. Re-running is both non-deterministic and potentially destructive — the trace is the source of truth, the way a flight recorder is.

Memory check — 3 questions

The single most important field to capture per step, and why? → The assembled context the model saw — without it you can’t reproduce or explain the decision.
How do traces let you separate a retrieval failure from a reasoning failure? → If the context was wrong/missing → retrieval failure; if context was right but the decision wrong → reasoning failure.
What property must the audit trail have, and for what? → Immutable / append-only — for compliance, abuse detection, and trustworthy post-mortems.

Production Architecture Reviews

The same lens on five real systems. Reviewing by class beats reviewing by brand — the failure modes cluster by autonomy, blast radius, and how the system holds state.

Reviewed as archetypes, not from privileged internals. Claude Code and Cursor are public coding agents (see Lesson 3); “OpenClaw-style” stands in for the autonomous computer-use / browser-agent class; the support and billing agents are common business designs. Treat specifics as illustrative.

System (class)	Strengths	Weaknesses / failure modes	Scalability & ops risk
Claude Code coding agent, human-present	Tight HITL (edits visible, approvals); local execution; rich tools; user supplies context	Destructive shell/file actions; prompt injection via repo/web content; long tasks saturate context	Per-user, so scaling is the model API + rate limits; cost is the user’s; main ops risk is sandbox escape
Cursor IDE coding agent	Deep editor context (open files, LSP); fast inline loop; retrieval over the repo	Wrong-context edits; retrieval recall misses; multi-file changes hard to verify	Index freshness at repo scale; latency budget; cost per active developer
OpenClaw-style autonomous computer-use / browser	Operates real UIs with no API; broad reach across web apps	Highest blast radius (clicks real buttons); brittle to UI drift; prime prompt-injection target via page content	Hard to sandbox; flaky → retry storms; auditing GUI actions is painful; ops risk is acting on the wrong account
Enterprise support agent RAG + actions, semi-auto	Deflects volume; consistent answers; escalates the hard cases	Hallucination outside KB; per-tenant data leakage; injection via ticket/page content	Multi-tenant isolation & cost attribution; KB freshness; escalation-queue backpressure
Billing agent financial writes, gated	Automates credits/retries/invoices; clear ROI; auditable	Double-charge on non-idempotent retry; compliance exposure; wrong-customer action	Correctness & idempotency are non-negotiable; immutable audit; reconciliation; approval gates above a threshold

🧮 Memory rule: Review by class, not by brand. Plot each system on autonomy × blast radius × statefulness — the controls it needs fall out of where it lands.

Memory check — 2 questions

Which archetype has the highest blast radius and why? → The autonomous computer-use/browser agent — it acts on real UIs with no API guardrails and is a prime injection target via page content.
What is the non-negotiable control for a billing agent? → Idempotency on every write (+ immutable audit + approval gate above a threshold) — to prevent double-charges.

Design Patterns — the reusable shapes

Six patterns cover most production agents. Like GoF patterns, the skill is knowing when not to use one. Each is a named answer to a recurring structural problem.

PlannerSeparate “decide the steps” from “do the steps.” Use when tasks are decomposable and you want to inspect/approve the plan. Skip for trivial one-shot tasks.

ExecutorA focused runner for one step type with tight tools + validation. Use to isolate side-effecting work behind a clean boundary. Skip if there’s only one tool.

SupervisorAn overseer monitors progress, enforces budgets, and can halt/redirect a worker. Use for long/expensive runs. Skip for short, cheap, low-risk tasks.

ApprovalInsert a human gate before irreversible/high-blast actions (M6). Use whenever blast radius is high. Skip for fully reversible, low-value actions.

Retriever (RAG)Fetch → assemble → ground the model in external knowledge. Use when answers depend on data outside the weights. Skip when the model already knows or inputs are self-contained.

Research / ReflectionFan out parallel searchers, then synthesize; optionally an evaluator critiques and the agent revises. Use for broad, open-ended discovery. Skip for narrow lookups (overkill + cost).

Pattern	Solves	Don’t use when
Planner	Open-ended task needs structure / approval	Steps are fixed → use a workflow
Executor	Isolate + validate side-effecting steps	Single trivial tool call
Supervisor	Long/expensive runs need a governor	Short cheap tasks (pure overhead)
Approval	Irreversible / high-blast actions	Fully reversible low-value actions
Retriever	Answers depend on external/fresh data	Self-contained input or known facts
Research/Reflection	Broad discovery; quality needs a critic	Narrow lookup — cost > benefit

🧮 Memory rule: Patterns are answers to structural problems — name the problem first. If you can’t state the problem a pattern solves here, you don’t need the pattern yet.

Memory check — 2 questions

Which pattern separates deciding steps from doing them, and what does that buy you? → Planner — inspectable/approvable plans, parallelizable steps, and a clean re-plan boundary.
When is the Research/Reflection pattern the wrong choice? → Narrow lookups — the parallel fan-out + synthesis + critique costs far more than the task is worth.

Anti-Patterns — how agent projects actually fail

None of these fail in the demo. They fail in week three of production — quietly, then expensively. Each is a default you have to actively resist.

Agent for everythingUsing a non-deterministic loop for work an if/else or a workflow would do. Consequence: pay latency, cost, and flakiness for control flow you didn’t need.

Multi-agent overuseSplitting sequential, shared-state work across agents. Consequence: coordination overhead, dropped context at handoffs, N× cost, “which agent broke?” debugging.

Unlimited autonomyNo gates on irreversible actions. Consequence: one bad decision × broad permissions = the incident that ends up in the post-mortem and the press.

Poor contextDumping everything (or the wrong things) into the window. Consequence: lost-in-the-middle, context poisoning, high cost, confidently wrong answers.

Missing observabilityNo per-step traces / assembled-context capture. Consequence: “can’t reproduce it,” no root-cause, no audit, every incident is a guess.

No evaluation”We test it by trying it.” Consequence: silent regression from a one-line prompt tweak; no way to ship changes safely or prove improvement.

No guardrailsLimits live in the system prompt (“please don’t loop”). Consequence: runaway loops, unbounded spend, prompt-injected actions — because the prompt is a suggestion, not a control.

Autonomy theatreDemo-grade autonomy shipped as production. Consequence: 99% per-step × 40 steps ≈ 67% end-to-end — quietly unreliable, eroding trust with every run.

✗ “Guardrails are in the system prompt.”

✓ A prompt is a request the model may ignore (or be injected past). Real guardrails — caps, budgets, permissions, idempotency, gates — live in the runtime where they’re enforced, not asked for.

🧮 Memory rule: Every anti-pattern is a default you must resist. Agents, more agents, more autonomy, more context, and “the prompt will handle it” all feel like progress and usually aren’t.

Memory check — 3 questions

Why is “guardrails in the system prompt” an anti-pattern? → The prompt is advisory and injectable; enforcement (caps, budgets, permissions, idempotency, gates) must live in the runtime.
What’s the consequence of the “agent for everything” anti-pattern? → Paying non-determinism, latency, and token cost for control flow a workflow or if/else would handle deterministically.
Why does “autonomy theatre” fail on long tasks specifically? → Per-step errors compound: ~0.99^40 ≈ 67% end-to-end, so long autonomous chains are quietly unreliable.

★ Executive summary

What you now understand (Phase 4)

Fourteen modules, one idea: an agent run is a long-lived, non-deterministic distributed computation. Three things to carry into every review:

The runtime is the execution engine, not the model. Planning, state, the loop, recovery, durability, cost, and observability all live around the LLM. When an agent behaves badly in production, look at the runtime first.
Every failure mode has a precedent. Retries, fallbacks, sagas/compensation, circuit breakers, dead-letter queues, checkpointing, durable execution — you already know this from distributed systems. Apply it; don’t reinvent it.
Autonomy is earned per action, by blast radius. Guardrails belong in the runtime, success needs an external verifier, and “fully autonomous over a long task” is quietly unreliable (0.99⁴⁰ ≈ 67%).

The one diagram to draw

Goal → Planner (decompose + re-plan) → Execution engine (sequential / parallel / branching) → Loop (observe→think→act, runtime stop-gates) → over durable state (checkpoint + resume), with recovery (retry→fallback→compensate→escalate), HITL gates on irreversible actions, and traces + budgets wrapping the whole thing. That’s a production agent.

★ Retrieval practice — Modules 1–14

Q1. Agent cost grows super-linearly with step count primarily because...

Q2. The safe way to recover an agent that already issued a wrong, irreversible refund is...

Q3. The most reliable way to know an agent task actually succeeded is...

Q4. A multi-hour agent must survive a deploy that restarts its process. The right model is...

Q5. To answer "why did the agent do that?" three days later, you must have captured...

★ Interview — pick your bar

Answer out loud in ~60s, then reveal. Core = recall · Senior = trade-offs & failure modes · Staff = synthesis under ambiguity · System Design = open design round (a different axis, not a harder level).

Define agent systems engineering and name its core concerns.

Hit these points: agent engineering = make one agent work (prompt + tools + loop) → agent systems engineering = run it as a production system → core concerns: planning, state management, execution engine, loop control, human-in-the-loop, failure recovery, orchestration, long-running durability, cost, observability → the through-line: an agent run is a long-lived, non-deterministic distributed computation, and all of those follow from that.

Walk through the agent loop and say exactly when it stops.

Hit these points: observe (gather results/context) → think (model decides next action) → act (tool call) → repeat → completion stops: model emits a final answer with no tool call, or calls an explicit done/submit tool, ideally confirmed by an external verifier → safety stops (runtime): step cap, wall-clock timeout, token/cost budget, repetition detector, human interrupt → the model proposes stopping; the runtime decides — stops are enforced outside the prompt.

What is agent state, and why is the LLM called stateless?

Hit these points: state = goal/current task + the plan + message history + tool outputs/intermediate results → the LLM is a pure function f(input)→output with no memory between calls → the runtime makes it stateful by re-assembling and re-sending context every turn → the context window is the slice of state the model sees this call → externalize state and you can scale, checkpoint, and recover; trap it in process memory and you can't.

Name the four rungs of agent failure recovery and a one-line example of each.

Hit these points: retry — transient idempotent tool error, backoff + jitter, bounded → fallback — degrade to a smaller model / cache / alternate tool → compensate — irreversible side effect already happened, run a registered undo (Saga) → escalate — recovery exhausted, dead-letter + human → climb only as far as needed, and wrap tools in circuit breakers.

Why is success detection hard, and how do you make stopping reliable?

Hit these points: the loop's natural stop is the model self-reporting "done," and the model is frequently wrong — it quits early on unfinished work or loops when finished → so never trust self-report alone: add an external verifier (tests pass, schema validates, artifact exists) → and add runtime safety stops independent of the model (step cap, wall-clock timeout, budget, repetition detector, human halt) → the senior framing: completion is a verification problem, termination is a control problem — solve them separately, both in the runtime.

Why is multi-agent so often the wrong call, and how do you decide?

Hit these points: teams reach for it for prestige or to "add intelligence," but intelligence is a model/context lever, not a topology lever → each split buys coordination overhead, a context handoff that can drop information, harder debugging ("which agent broke?"), and N×M cost → decide like a microservice split: only when work is genuinely independent & parallel, context is separable and exceeds a window, or subtasks need distinct toolsets → default to a single agent (or a router) and earn each split with a real boundary.

Map agent failures onto distributed-systems patterns — and name the one agents get wrong.

Hit these points: tool failure = downstream service failure → retry + circuit breaker; model failure = bad RPC response → validate + repair + retry on another tier; context failure = cache pollution → compress/re-rank/re-inject; planning failure = bad query plan → re-plan; execution failure = partial write → saga / idempotency → the one agents get wrong is compensation: there's no rollback for a sent email or an issued refund, so every irreversible action needs a registered compensating action — plus idempotency keys upstream so retries don't double-apply.

An agent's bill 10×'d overnight. Diagnose and design the controls.

Hit these points: first suspects — a runaway loop (no progress, repeated calls), context bloat (history re-sent and growing), or a multi-agent fan-out amplifying both → cost is super-linear because input grows each step, so the unit to watch is cost per completed task, attributed per tenant → controls: prompt caching on the stable prefix, history compaction, model tiering, per-task step + token budgets with a kill switch, and an anomaly alert at ~3× baseline so it trips an incident before the invoice → close the loop with the runtime guards from loop control so "runaway" can't happen silently.

Design the execution + durability layer for an agent that runs for hours and survives crashes.

Hit these points: model the run as a durable workflow, not an in-memory loop — the agent loop is the non-deterministic step inside deterministic orchestration → state (work-list with per-item status, plan, cursor, artifacts) lives in a durable store; checkpoint after each meaningful item → on restart, resume from the last checkpoint via deterministic replay, skipping completed items → every replayed/ retried step is idempotent (keys) so resume can't double-charge or duplicate → layer recovery (retry→fallback→compensate→escalate), HITL gates before irreversible writes, per-item budgets, and per-item traces + an immutable audit log → name the through-line: borrow durable-execution discipline (Temporal-style) so a non-deterministic worker gets deterministic retries, checkpoints, and observability.

An engineer wants to convert a stable, well-defined business process into an agent. Make the call.

Hit these points: if the steps are known and rarely change, an agent is the wrong tool — you'd pay non-determinism, latency, and token cost for control flow a workflow or plain code handles deterministically, debuggably, and testably → "agent for everything" is the dominant anti-pattern → the senior move: put it on the workflow↔agent spectrum — fixed path → workflow; open-ended path the model must decide → agent; hybrid → workflow with agent steps only where the path is genuinely unknown → reserve agency for the irreducible uncertainty and keep the rest deterministic → this is also cheaper to operate and far easier to get past a review.

"Ship it fully autonomous." Make the principal-level call on autonomy and reliability.

Hit these points: reframe autonomy as earned per action by blast radius, not a global switch → reads/analysis auto; irreversible or high-impact writes behind HITL gates until data justifies loosening → do the compounding math out loud: 99% per step over 40 steps ≈ 67% end-to-end, so long fully-autonomous chains are quietly unreliable (illustrative) → price in the hidden costs: supervised agents still need a human, production error > demo error, token cost isn't linear → the position: start supervised on a bounded action set, instrument success rate and cost per task, widen autonomy only where measured reliability + bounded blast radius support it — with an audit trail and kill switch throughout.

Design-round framework — drive any "design a production agent system" prompt through these, out loud:

Clarify the job: responsibilities, success metric, and the risk profile of its actions.
Agent or workflow? Put it on the spectrum — reserve agency for genuinely open-ended paths; keep the rest deterministic.
Planning & execution: decomposition, re-plan triggers, and the execution shape (sequential / parallel / branching).
State & durability: externalized state, checkpoint granularity, resume-by-replay, idempotent steps.
Loop control: completion (external verifier) + safety stops (cap, timeout, budget, repetition, halt).
Recovery: retry → fallback → compensate → escalate; circuit breakers; compensation for irreversible actions.
Autonomy & HITL: gates by reversibility × blast radius; escalation/dead-letter path.
Cost: caching, compaction, tiering, per-task budgets + kill switch, per-tenant attribution + anomaly alert.
Observability: per-step traces with assembled context, immutable audit log, evals gating prompt/model changes.

Design an autonomous incident-investigation agent that runs for days across on-call handoffs.

A strong answer covers: scope it — investigate a bounded class of incidents, success = faster correct root-cause without causing new incidents → durable workflow spanning days/hosts/handoffs: state (timeline, hypotheses, evidence, current lead) in a durable store, checkpointed; resumable by any worker → execution: parallel evidence-gathering (logs/metrics/traces) → synthesize → branch on hypothesis; re-plan when evidence kills a lead → autonomy: reads auto; any mitigation that writes to prod is HITL-gated with blast-radius estimation; nothing destructive auto-fires → recovery: tool failures retried/circuit-broken; the run never silently dies — it checkpoints and escalates → cost: per-investigation budget + alert, cheap model to triage, strong model for reasoning → observability: every step traced with assembled context; an immutable incident timeline that doubles as the audit + post-mortem → handoff: a human can read the state, take over, or hand back at any checkpoint → name the trade-off: days-long autonomy risks drift and stale context, so periodic goal re-injection + human checkpoints keep it anchored.

Design a billing-operations agent that applies credits, retries charges, and updates plans — safely.

A strong answer covers: correctness is non-negotiable — this writes money → single agent default (mostly sequential, shared state); fan out only for independent reads → tools split by risk: read-only (usage, invoices) broad; writes (credit, charge, plan change) least-privilege and idempotency-keyed so a retry is a no-op → HITL gates: credits/refunds above a threshold and any plan downgrade require approval → recovery: charge failed-but-maybe-applied → idempotent retry; wrong charge that landed → compensating reversal (Saga); never blind-retry a non-idempotent write → durability: each operation checkpointed; on crash, resume without re-applying completed writes → cost/obs: per-task budget, immutable audit log of every monetary mutation (who/what/when/amount/idempotency key), per-tenant attribution → injection defense: ticket/customer text is data, never instructions → name the trade-off: automation cuts toil and is fully auditable, but every irreversible monetary action must be idempotent and gated, or one retry becomes a double-charge incident.

★ Agent systems engineering cheat sheet

Concept → engineering translation

Agent concept	Engineering equivalent	One-line test
Planning	Query planner / decomposition	”Can it replace the stale tail of the plan?” If no, it’s a script.
State	Server-side session over a stateless handler	”Remove the re-sent history — does memory vanish?” Yes.
Execution engine	DAG executor (drawn at runtime)	“Who defines the graph & when?” Model-at-runtime = agent.
Loop control	while-loop + watchdog + circuit breaker	”What guarantees it terminates?” Runtime caps, not the prompt.
Success detection	External verifier / assertion	”Did a check pass, or did the model just say so?”
Failure recovery	Retry / fallback / saga / DLQ	”Is there a compensation for irreversible actions?”
Long-running	Durable execution (Temporal)	“Does it resume from a checkpoint or restart from zero?”
HITL	Approval gate / IAM step-up	”Does a human gate irreversible, high-blast actions?”
Cost	Per-task budget + attribution	”What’s cost per completed task, and the kill switch?”
Observability	Distributed tracing + audit log	”Can you replay the assembled context of any step?”

Five rules to carry into any agent review

The runtime is the execution engine, not the model. Bad production behavior → look at the runtime first.
The plan is a hypothesis; re-planning is the algorithm. No re-plan path = brittle script.
The model proposes stopping; the runtime decides. Verify success externally; enforce safety stops independently.
Every failure mode has a distributed-systems twin. Apply the old theory; the only new trap is compensation.
Autonomy is earned per action by blast radius. Guardrails live in the runtime; long full-auto chains compound to unreliable.

★ Catalogs & checklists

The portable artifacts — print these. The unifying lens:

an agent request is a long-lived, stateful, non-deterministic session over privileged tools.

Almost every line below follows from that.

Architecture review checklist

Is this even an agent? Could a workflow / plain code do it deterministically?
Planning: is there a re-plan path when reality diverges, or is the plan a one-shot script?
State: is it externalized so any worker can resume, or trapped in process memory?
Execution: where does parallelism live, and is dependency/ordering explicit?
Loop: step cap + wall-clock + budget + repetition enforced by the runtime?
Success: is there an external verifier, or does the agent self-certify?
Recovery: retry/fallback/compensation/escalate mapped per failure type?
Autonomy: gates by reversibility × blast radius; irreversible actions never auto-fire?
Durability: checkpoint + resume; replayed steps idempotent?
Observability: per-step traces with assembled context; immutable audit log?
Cost: per-task budget + kill switch; per-tenant attribution + anomaly alert?
Evals: a harness that gates prompt/model changes against regression?

Production readiness checklist

Termination guaranteed by runtime caps (step + wall-clock), not the prompt.
Budget + kill switch per task and per tenant; anomaly alert at ~3× baseline.
Idempotency keys on every mutating tool; retries can’t double-apply.
Compensation registered for every irreversible action.
Circuit breakers + bulkheads + timeouts per tool and per provider.
Checkpoint + resume for any task > a few minutes; tested by killing it mid-run.
HITL gates on irreversible / high-blast actions; escalation/dead-letter path exists.
Per-step traces (assembled context + output + tokens + cost + status) retained.
Immutable audit log for every mutation; PII/secret redaction in logs.
Prompt-injection defense: tool/RAG outputs tagged as data, never instructions.
Least privilege per agent/tool; read and write paths separated.
Eval set + canary/rollback for prompt and model changes.

Failure mode catalog

Failure	Trigger	Control
Runaway loop	No progress / repeated calls; no cap	Step cap + wall-clock + repetition detector
Cost explosion	Re-sent history; multi-agent fan-out	Caching, compaction, tiering, budget + kill switch
Double side effect	Non-idempotent retry / resume	Idempotency keys; checkpoint before write
Unrecoverable side effect	Irreversible action then failure	Compensation (Saga); HITL gate upstream
Context poisoning	Stale/wrong data accumulates	Compaction, re-rank, goal re-injection, pruning
Lost-in-the-middle	Over-stuffed window	Context budget per section; retrieve, don’t dump
Prompt injection	Adversarial tool/RAG content	Tag outputs as data; least privilege; sandbox
Silent regression	Prompt/model tweak, no eval	Eval harness gating changes; canary + rollback
”Can’t reproduce”	No trace / no assembled context	Per-step traces; immutable audit log
Lost work on crash	In-memory state, long task	Durable workflow: checkpoint + resume
Compounded unreliability	Long full-auto chain	HITL gates; shorten chains; external verifiers
Noisy neighbor	One tenant starves quota/cost	Per-tenant isolation, quotas, attribution

Design pattern catalog

Planner (decide vs do · skip if steps fixed) · Executor (isolate side-effecting steps · skip if one tool) · Supervisor (govern long/expensive runs · skip if short) · Approval (gate irreversible actions · skip if reversible) · Retriever/RAG (ground in external data · skip if self-contained) · Research/Reflection (parallel discovery + critic · skip narrow lookups) · Router (classify & dispatch · skip if one path) · Coordinator+Workers (parallel/separable work · skip if sequential shared-state).

Anti-pattern catalog

Agent for everything (workflow would do) · Multi-agent overuse (sequential shared-state work split into agents) · Unlimited autonomy (no gates on irreversible actions) · Poor context (dump-everything → poison + cost) · Missing observability (no traces → “can’t reproduce”) · No evaluation (silent regressions) · No guardrails (limits live in the prompt) · Autonomy theatre (demo-grade shipped as production).

VP Engineering review questions

“Why is this an agent and not a workflow? What’s the irreducible uncertainty agency buys us?”
“What’s the per-step success rate, and therefore the end-to-end rate over a full task?” (0.99⁴⁰ ≈ 67%.)
“What guarantees the loop terminates — show me the runtime cap, not a prompt sentence.”
“Enumerate every irreversible action. For each: what’s the gate, and what’s the compensation?”
“Where does state live if a worker dies at step 40 of 60? Walk me through the resume.”
“What’s cost per completed task, the kill switch, and the per-tenant anomaly alert?"
"Show me how you debug a wrong action from three days ago without re-running it.”
“I change one prompt line — what automatically catches a 5% regression before it ships?"
"Why this topology? Justify every agent split like a microservice boundary.”

★ Revision guides & what to learn next

5-minute weekly recall

Recite from memory, no looking:

The one-line definition: an agent run is a ______ computation. (long-lived, non-deterministic, distributed.)
Draw the production-agent diagram: Goal → Planner → Execution engine → Loop, over durable state, with recovery + HITL + traces.
Two stop categories and the members of each (completion vs safety).
The four recovery rungs in order, and the one agents get wrong.
Why cost is super-linear, and the two levers that flatten it.
The microservice test for “is multi-agent justified?”

Stuck on one? Re-read that module’s .rule band, close it, try again. Recall > re-reading.

30-minute deep dive (before a big review)

Planning & loop — Read the ReAct and Plan-and-Solve abstracts; from memory, draw the plan→execute→re-plan cycle and label where each lives.
Patterns — Re-read Anthropic’s “Building effective agents”; map prompt-chaining, routing, parallelization, orchestrator-workers, evaluator-optimizer to Modules 8 & 13.
Durability — Read the Temporal “durable execution” overview; restate checkpoint/replay/idempotency in agent terms (M9).
Observability — Skim the OpenTelemetry GenAI semantic conventions; list the span attributes you’d require per step (M11).
Apply it — Take one system from Module 12 and run the full architecture-review checklist on it out loud. Could you defend it to a CTO?

Roadmap — what to learn next

Topic	Why next	Start here
Agent evaluation & testing	The hardest unsolved production problem — you can’t ship changes safely without it. The natural Phase 5.	Anthropic eval guide; Braintrust; LangSmith
Durable execution in depth	The backbone of long-running agents; learn it as a primitive, not a library.	Temporal docs; “durable execution” patterns
Agent security (deep)	Prompt injection, tool abuse, data exfiltration scale with autonomy.	OWASP LLM Top 10; Anthropic RSP
Context engineering at scale	Retrieval, ranking, compaction are where production leverage and cost live (Phase 3, deeper).	Revisit Lesson 2; pgvector + rerankers
Build one end-to-end	Nothing cements it like shipping. Build the billing or investigation agent with real durability + traces.	Claude API + MCP SDK + a workflow engine

← Lesson 4: Multi-agent Systems & Production

Phase 4 complete ✓

Was this lesson helpful? Thanks — noted.

These notes reflect my current understanding and are updated as I learn, build, and discover better explanations.