The agent harness.

Key terms

Agent harness

Everything around the model that turns it into a working product. Also called scaffolding.

Context engineering

Deciding, at every step, which information goes into the model’s context window and which stays out.

Compaction

Summarizing an older part of a conversation and continuing with the summary, so a long task fits in the context window.

Memory

Information saved outside the context window and brought back in a later turn or session, such as a shopper’s saved room size.

State

Where a task stands right now: the plan, what’s done, what’s next, and the results so far.

Checkpoint

A planned pause where the work is saved, and often where a person reviews it before the agent goes on.

Stopping condition

A rule that ends a run: the job is done, a limit is reached (such as a maximum number of steps or a spending limit), or a person is needed.

The idea

When an AI feature fails, the first instinct is to blame the model and try a newer one. Sometimes that helps. Often it doesn’t, because the model was never the problem. The shopper in the opening story told the assistant her door width. The harness just never put it back in front of the model.

This lesson brings the course together. You’ll walk one shopper’s question through every part you’ve learned, then go deeper on what only this lesson covers: context, memory, long tasks, build or buy, and launch readiness. The furniture store is still made up.

The lesson, in seven parts

1. 1What the harness is, and why it matters as much as the model.

   In lesson 1 you saw that an AI product is a model plus everything your team builds around it. That “everything around it” has a name: the harness. Some engineers call it the scaffolding. Anthropic’s Applied AI team now uses a short definition of an agent: “LLMs autonomously using tools in a loop.” The harness supplies the loop, the tools, and everything else the model can’t do on its own.

   Here’s why that matters to you. In January 2025, Anthropic wrote about SWE-bench, a public test of real software engineering tasks that scores the model and its scaffolding together. Their words: “The performance of an agent on SWE-bench can vary significantly based on this scaffolding, even when using the same underlying AI model.” Same model. Different harness. Different results.

   Erik Schluntz and Barry Zhang, describing an agent they built for that test, wrote that they “actually spent more time optimizing our tools than the overall prompt.” Small harness decisions add up. Here are the parts:

   Part	At the made-up furniture store	Lesson
   The loop	Keeps going until both questions are answered	2
   The planner	“Check the door, stock, and delivery, then answer”	2
   Tools	Doorway-size check, stock check, delivery date check	3
   Subagents	A helper that compares three sofas	4
   Memory and state	The shopper’s saved door width and shortlist	9
   Context management	The Harbor’s page goes in; the catalog stays out	1 and 9
   Guardrails	No delivery date the check didn’t return	6
   Evals	Real questions with known-good answers	5
   Observability	Every step of every answer, saved as a trace	7
   Cost and speed	Simple questions go to a smaller model	8

   Made-up example. Teams group these parts differently, and some count subagents and cost controls as part of the loop; what matters is that someone owns each one.

   Look at the opening story again. The model could answer the doorway question. Memory and context management failed to give it the door width. That’s a harness bug, and no model upgrade would fix it.

2. 2One question, through every part.

   Now walk the shopper’s question through a harness that works, with the lesson for each part:

   • The shopper asks. She’s signed in, so the harness knows who’s asking.
   • It builds the context (lessons 1 and 9): the system prompt, her saved door width, the Harbor’s product page, and the last few turns. The full catalog stays out.
   • The model plans (lesson 2). Two questions need two checks: will it fit, and can it arrive by Saturday?
   • It calls tools (lesson 3): the doorway-size check, then the stock and delivery date checks. Each result goes back into the window, and the loop repeats until both are answered.
   • Guardrails check the draft (lesson 6). Is every date one the delivery check returned? Is every measurement from the product page? Is anything private showing?
   • It answers: “The Harbor won’t fit through a 30-inch door, but the Harbor Modular will. The Harbor Modular’s earliest delivery to you is Monday, not Saturday.”
   • The trace is recorded (lesson 7): every step, tool call, and result, so a wrong answer can be traced to where it went wrong.
   • Evals score it (lesson 5). Samples of real traces are checked against your criteria, by people or an automated grader, and failures become new test cases with known-good answers.

   Under all of it runs lesson 8: every step costs tokens and time. If your team uses an AI gateway, every model call in this walk-through passes through it. When something goes wrong, this walk-through is your map. Ask which step it went wrong in.

3. 3Context engineering: what goes into the window.

   In September 2025, Anthropic’s Applied AI team described context engineering as “the set of strategies for curating and maintaining the optimal set of tokens (information) during LLM inference.” In plain words: at every step, someone decides what the model gets to see. That’s the prompt, plus tool results, memory, documents, and the conversation so far.

   Why not send everything? Lesson 1 introduced context rot: as the window fills, recall gets worse. The same team says models have an “attention budget,” and every token spends some of it. Their principle: find “the smallest possible set of high-signal tokens that maximize the likelihood of some desired outcome.” Each piece of information ends up in one of three places:

   • In the window now: what this step needs. For the doorway question, the Harbor’s measurements and the door width, not the whole catalog.
   • Shortened: compaction summarizes an older stretch of conversation and carries on with the summary. Anthropic warns that overly aggressive compaction can lose “subtle but critical context whose importance only becomes apparent later.” A lighter touch is clearing old tool results the model has already used.
   • Fetched only when needed: the article calls this “just in time.” The agent keeps a pointer, like a product ID, and loads details with a tool when a step needs them. It saves room but is slower, so some teams load the basics up front and fetch the rest.
   • A question for your engineers: “For our five most common questions, what exactly is in the window?” If nobody knows, start there.

4. 4Memory across sessions: what to keep, what to forget.

   A model doesn’t remember last week on its own. Memory is something the harness builds: it saves information outside the window and puts the right piece back when it’s needed. In the opening story, the door width was saved but never put back.

   Anthropic’s memory tool documentation shows one common pattern. The model asks to save or read a note, and your application does the storing: “You control where and how the data is stored through your own infrastructure.” Its safeguards are your job too, like stripping sensitive data before it’s written and deleting memories nobody has used in a long time.

   Privacy law points the same way. In the European Union, the GDPR says personal data should be “limited to what is necessary” and kept no longer than necessary, and in certain cases people can have it erased. Rules differ by country, so check with your privacy and legal team. A useful test: does it help the shopper next time, and would they expect you to keep it?

   Information	A sensible choice	Why
   Door width and room size she saved	Keep, and show it to her	She chose to share it, and it saves measuring again
   Her sofa shortlist	Keep, with a way to clear it	Lets her pick up where she left off
   The full chat transcript	Don’t keep in memory	Most of it won’t help next time
   Payment card details	Never put in assistant memory	Not needed for advice, and a serious risk if exposed
   A guess about her income	Don’t store	She didn’t share it and wouldn’t expect it kept

   Made-up example. Check your own rules with your privacy and legal team.

   Two more product decisions: can shoppers see and delete what’s remembered? And when a memory may be stale, like a door width from before they moved, does the assistant confirm it first? “You told me your door is 30 inches. Is that still right?” prevents a big mistake.

5. 5Long-running tasks and checkpoints.

   Not every job fits in one answer. Say the made-up store asks an agent to check all 3,000 product pages for missing or conflicting measurements. That takes many context windows. In a November 2025 Anthropic article, Justin Young names the core problem: each new session “begins with no memory of what came before,” like engineers working in shifts who arrive knowing nothing of the last shift.

   His team saw three failures: the agent tried to do too much at once and ran out of room; a later session saw progress and declared the job done; and work was marked complete without proper testing. Summarizing alone wasn’t enough. What helped:

   • Write the full list of work first, with every item marked not done, so no session can declare victory early.
   • Work on one item at a time.
   • End every session by writing progress notes and saving the work, so the next session can read what happened.
   • Start every session by reading those notes and running a quick check that nothing is broken.

   Schluntz and Zhang add two controls. Agents can “pause for human feedback at checkpoints or when encountering blockers,” and teams often add stopping conditions, like a maximum number of steps. At the store, a person approves every measurement fix before it goes live, and each run stops after a set number of pages or a set spend (lesson 8). You decide where the pauses go. A good rule of thumb: pause before anything a customer will see, or anything hard to undo.

6. 6Build, buy, or something in between.

   Sooner or later someone asks: build our own harness, or buy one? It’s rarely all or nothing. You’re choosing how much of the harness your team writes.

   Schluntz and Zhang note that frameworks make it easy to start, but “they often create extra layers of abstraction that can obscure the underlying prompts and responses, making them harder to debug.” They suggest starting with the provider’s API directly, and understanding the code under any framework you use. Anthropic’s Applied AI team puts it simply: “do the simplest thing that works.”

   Option	What you get	What to watch
   Buy a ready-made assistant	The fastest start	Can you see traces, run your evals, and control guardrails and memory?
   Build on a framework or agent SDK	A head start on the loop and tools	Does your team understand what it does underneath?
   Build directly on a model API	The most control and visibility	More engineering time; you own every part

   General trade-offs, not a ranking. It depends on your team, timeline, and how central the feature is.

   Whichever you choose, your product data, evals, guardrail decisions, and the launch call stay yours. When a vendor demo looks great, run it on your own questions and ask to see the trace of a wrong answer. If you can’t see why it was wrong, you can’t fix it.

7. 7A launch-readiness checklist.

   Bring this checklist to a launch review. You don’t need perfect answers. You do need an answer, and an owner, for each line.

   Ask

   • The customer problem it solves is written down, with the product metric that will show it helped and when you’ll check it after launch (lesson 1).
   • The job and the never-do list are written down (lessons 1, 6).
   • Facts come from data someone owns (lesson 1).
   • The loop has stopping conditions; long tasks have checkpoints (lessons 2, 9).
   • Risky tool actions need a person’s approval (lesson 3).
   • Evals on real questions pass the bar agreed in advance (lesson 5).
   • Guardrails are tested against expected attacks and mistakes (lesson 6).
   • Every answer leaves a trace, and someone reviews them (lesson 7).
   • Cost and wait per answer are inside your limits (lesson 8).
   • What’s remembered is visible and deletable (lesson 9).
   • There’s an off switch, and a person customers can reach.

   Avoid

   • “It worked in the demo.” (A demo is a handful of answers, not your customers’ thousands.)
   • “We’ll add evals after launch.” (Then nobody can say what “good enough” meant.)
   • “Let’s remember everything, just in case.” (More memory means more risk and more noise in the window.)

   A checklist doesn’t make a product safe on its own. It makes sure the right questions were asked before customers find the gaps.

With AI: Harness work is where AI can help you most as a product manager. Paste a trace of a bad answer into an assistant and ask it to point to the step where things went wrong, or ask it to walk your feature through the launch checklist and list what’s unknown. Then check its reasoning against the trace yourself. It can find likely suspects quickly; you still decide what’s true.

HARNESS MAP

The customer's question (in their words):

Context:
  In the window now:
  Summarized or cleared:
  Fetched only when needed:

Plan and loop steps:
Stopping condition (steps, time, or spend):

Tools, and which need a person's approval:

Memory:
  Keep:
  Never keep:
  How customers see and delete it:

Long tasks: where progress notes live; checkpoints:

Build or buy: can we see traces, run our evals, set our guardrails?

LAUNCH CHECK (ready / not ready / unknown, owner)
  Job and never-do list (L1, L6):
  Data owned and correct (L1):
  Loop limits and checkpoints (L2, L9):
  Tools described; risky actions approved (L3):
  Evals pass the agreed bar (L5):
  Guardrails tested (L6):
  Traces recorded and reviewed (L7):
  Cost and wait inside limits (L8):
  Memory visible and deletable (L9):
  Off switch and a human to reach:

Verdict:

Ask AI about this lesson

All lessons