How an AI product works.

Key terms

Large language model (LLM)

A model trained on very large amounts of text that generates text: it answers questions, summarizes, drafts, and follows instructions.

Token

The unit a model reads and writes: a word, part of a word, or a character. Usage, limits, and pricing are counted in tokens.

Context window

Everything the model can see while it writes one answer, including the answer itself. Its working memory, separate from what it learned in training.

System prompt

Instructions your team writes that the model sees before every conversation: its job, its tone, and its rules.

Hallucination

A plausible-sounding answer that is wrong or made up.

Grounding (RAG)

Looking up relevant facts at the moment of the question and putting them in the context window, so the model answers from them. RAG stands for retrieval-augmented generation.

Temperature

A setting that controls how much randomness goes into the model’s word choices.

The idea

Most people meet AI as a chat box. You type, it answers, and it sounds sure of itself. So it’s easy to picture a very smart database somewhere, looking up the right answer. That picture is wrong, and it leads product teams into real trouble.

This lesson replaces it with a better one. You’ll see what the model actually does, why it can be confidently wrong, and what your team builds around it to make it useful. The rest of this course takes those parts one at a time. One made-up example runs through every lesson: an online furniture store adding an AI shopping assistant.

The lesson, in seven parts

1. 1What a large language model actually does.

   Start with how a model is made. First comes pretraining: the model reads a huge amount of text and learns to predict the next word, given the words before it. That’s Anthropic’s own description of how its models start out. A model at this stage isn’t good at answering questions or following instructions yet.

   So it’s trained further. Fine-tuning teaches it from more examples. Reinforcement learning from human feedback (RLHF) has people rank example answers, and the model learns to prefer answers like the higher-ranked ones. That’s what turns a next-word predictor into an assistant that follows instructions.

   Here’s the part that matters for you. When the model answers, it isn’t looking anything up. It writes a likely next piece of text, then the next, then the next. Often that text is right, because it learned a lot. But “likely” and “true” aren’t the same thing.

   Two more things follow from how it’s made. A model learns from text up to a date, its training data cutoff, and model providers publish that date for each model. It knows nothing after it unless you tell it. And it has never seen your store’s product catalog, your return policy, or today’s stock. Unless you give it those facts, it can only guess.

2. 2Tokens: the unit everything is counted in.

   Models don’t read words the way you do. They read tokens: words, parts of words, or single characters. How many characters fit in a token depends on the model and the language. Anthropic says a token was about 3.5 English characters on its earlier Claude models, and that its newer models split the same text into about 30 percent more tokens (as of October 2026). Other model makers split text their own way, so the same sentence can be a different number of tokens on a different model.

   Why should a product manager care? Because tokens are how the work is measured:

   What tokens affect	What it means for your product
   Cost	Model providers usually charge per token, for what goes in and what comes out. A longer prompt or a wordier answer costs more, on every single question.
   Speed	The model writes one token after another. Longer answers take longer to finish, and shoppers feel that wait.
   Limits	The context window holds a fixed number of tokens. Instructions, documents, the conversation, and the answer all have to fit.

   Prices and limits change often and differ by model. Check your provider’s current pricing and model pages, never a number you remember.

   You’ll come back to this in lesson 8, on cost, speed, and quality. For now, remember the shape of it: every word you put in front of the model, and every word it writes, has a price and a wait attached.

3. 3The context window: what the model can see right now.

   Anthropic defines the context window as all the text a language model can reference when generating a response, including the response itself. It calls it the model’s “working memory”, and it’s separate from everything the model learned in training.

   Picture the furniture store’s assistant answering that doorway question. In its context window, there might be:

   • The system prompt: “You’re the store’s shopping assistant. Answer only about our products…”
   • The conversation so far: the shopper’s question, and anything said before it.
   • Facts your team added for this question: the Harbor sofa’s product page, with its measurements.
   • Results from tools it used, like a delivery date check.
   • The answer it’s writing, as it writes it.
   • If a fact isn’t in the window, and the model didn’t learn it in training, the model doesn’t have it.
   • Bigger isn’t automatically better. Anthropic notes that as the token count grows, accuracy and recall degrade, and it calls this “context rot”. Choosing what goes into the window matters as much as how big it is.
   • Most models don’t remember last week’s conversation on their own. If your product “remembers” a shopper, your team built that, and decided what to put back in the window.

4. 4Why answers vary, and why models make things up.

   Ask the same question twice and you may get two different answers. That’s partly by design. Temperature is a setting that controls how much randomness goes into the model’s word choices: higher for more varied answers, lower for more predictable ones. Anthropic notes that even at a temperature of 0, results aren’t fully deterministic, and the same input can produce different outputs.

   So you can’t test an AI feature the way you test a button. You test it many times, on many questions, and judge the pattern. That’s what evals are for, and they get their own lesson (lesson 5).

   The bigger risk is the hallucination: an answer that sounds right and isn’t. In a 2025 paper, researchers at OpenAI and Georgia Tech (Kalai, Nachum, Vempala, and Zhang) argue that models guess when they’re uncertain because training and evaluation reward guessing over admitting uncertainty. Like a student on a multiple-choice test, a guess can score points, and under right-or-wrong scoring, “I don’t know” never does.

   Here’s how that plays out at the made-up furniture store:

   What the assistant had	What it said
   Only its training	“Yes, most sofas fit through a standard door.” Confident, general, and possibly wrong for this sofa.
   The Harbor sofa’s product page, with its measurements	“The Harbor’s diagonal depth, the measurement that has to clear the doorway when movers tilt it through, is 34 inches. That’s more than a 30-inch door, so it won’t fit. The two-piece Harbor Modular will.” Specific, and based on facts you can check.

   Made-up example. The sofa, its measurements, and both answers are invented for this lesson.

   Anthropic’s guide to reducing hallucinations suggests a few practical steps. Give the model explicit permission to say “I don’t know”. Ground its answers in documents you provide. For long documents, have it pull out direct quotes first. And ask it to cite the source behind each claim, so you can check it.

5. 5Grounding: give the model the facts.

   If the model doesn’t know your catalog, give it the right piece of the catalog at the moment of the question. That’s grounding. The best-known way to do it is retrieval-augmented generation (RAG). The name comes from a 2020 paper by Patrick Lewis and colleagues at Facebook AI Research, University College London, and New York University. Today it usually means what this part describes: look up the right text, and put it in the context window.

   In plain steps, for the doorway question:

   • The shopper asks: “Will the Harbor sofa fit through a 30-inch door?”
   • Your system looks up what’s relevant: the Harbor’s product page and its measurements.
   • Those facts go into the context window, next to the question.
   • The model answers from those facts, and can point to where each one came from.

   Here’s the part product managers often miss. Anthropic’s glossary puts it plainly: how well this works depends on the quality and relevance of the knowledge base, and of what gets retrieved. If the Harbor’s product page lists the wrong depth, the assistant will repeat the wrong depth, confidently. In AI products, your data is part of the product. Fixing a product page can matter more than changing the model.

6. 6The parts around the model, and who owns them.

   In a December 2024 guide, Anthropic engineers Erik Schluntz and Barry Zhang call the basic building block of agentic systems “the augmented LLM”: a model enhanced with retrieval, tools, and memory. The model is one part. The rest is built by a team, and a product manager shapes every one of those decisions.

   Here are the parts, what each one does, and where this course covers it:

   Part	What it does	At the made-up furniture store	Lesson
   The model	Reads the context window and writes the answer	A model from an outside provider	This one
   Instructions	Set the job, the tone, and the rules	“Answer only about our products. Never promise a delivery date you haven’t checked.”	This one
   Your data	Supplies the facts, at the moment of the question	Product pages, measurements, care guides, return policy	This one
   The loop	Lets the model plan, act, check, and try again	Find sofas, check sizes, compare, answer	2
   Tools	Let the model look things up and do things	Stock check, delivery date check, room planner	3
   Subagents	Split a big job across helper agents, each with its own focus	A sofa finder, a fit checker, and a delivery planner	4
   Evals	Decide whether answers are good enough to ship	A set of real shopper questions with known-good answers	5
   Guardrails	Stop what must never happen	No made-up measurements, no private data in answers	6
   Observability	Show what happened, so you can fix it	Every question, answer, and tool call, recorded and reviewable	7
   Cost and speed	Keep each answer affordable and fast enough	Choosing a model, keeping prompts lean	8

   The furniture store and its assistant are made up. Lesson 9 puts all the parts together.

   Notice how much of that table isn’t the model. When an AI feature disappoints, the fix is often in the data, the instructions, or the tests, not in switching to a newer model.

7. 7Questions to ask before an AI feature ships.

   You don’t need to train models to do this job well. You need to ask good questions early, and keep asking them after launch. Here’s a starting set:

   Ask

   • What job does it do for the customer, and what must it never do?
   • Where do its facts come from, and who keeps that data right?
   • What should it say when it doesn’t know?
   • How will we know an answer is good, before launch and after?
   • What does one answer cost, and how long will a customer wait for it?
   • When it’s wrong, how will we find out, and how fast can we fix it?

   Avoid

   • “Can we add AI to this?” (Start with the customer’s problem, not the technology.)
   • “Which model is the smartest?” (The best model for your job is the one that passes your tests at a cost you can afford.)
   • “The demo looked great.” (A demo shows a few answers. Customers will ask thousands of questions.)
   • “It sounds confident, so it’s probably right.” (Confidence isn’t evidence.)

   Each of these questions has a lesson coming. By the end of the course, you’ll be able to answer them for your own product, not just ask them.

With AI: AI is the subject here, so use it as a study partner. Ask an assistant to explain a term from this lesson with a new example, then check its answer against the sources below. That habit, using AI and verifying it, is the habit this whole course builds.

AI FEATURE BRIEF

The customer's problem (in their words):

The job (one sentence: what it does for them):

Facts it needs, and where each comes from:
  - Fact:            Source:            Owner:
  - Fact:            Source:            Owner:

Tools it can use (look up or do things):

It must never:
  1.
  2.
  3.

When it doesn't know, it says:

How we'll know an answer is good (before launch):

How we'll know it's working (after launch):

Cost and wait we can accept per answer:

Open questions:

Ask AI about this lesson

All lessons