Critique an app.

Key terms

App critique

A structured review of a product’s experience: what it’s for, where users struggle, and what to change first. Also a common product manager interview exercise.

Job to be done

The progress a person is trying to make in a particular situation. The Christensen Institute describes it as “the progress they’re trying to make as they strive toward a goal or aspiration within particular circumstances.”

Core journey

The few steps a user must take to get the app’s main job done, such as finding a sofa and buying it. Lesson 10 calls this a critical user journey.

Usability heuristic

A broad rule of thumb for interaction design. Jakob Nielsen’s 10 are the best known.

Heuristic evaluation

A method for finding design problems: evaluators judge an interface against a set of heuristics.

Severity rating

A score for how serious a usability problem is. Nielsen’s scale runs from 0 (not a problem) to 4 (a usability catastrophe).

The idea

Many people critique an app the way they’d review a restaurant. “The colors feel dated.” “I don’t like the menu.” “It should have dark mode.” Some of it may even be true. But it doesn’t tell anyone what to fix first, or why it matters.

A product manager’s critique starts somewhere else: what the app is for, and who it’s for. Then it follows one person through one task, notices where they get stuck, and explains why, using principles anyone can check. It ends with a short list of fixes, each with a cost and a way to know if it worked. That’s usually what interviewers are looking for when they say “critique an app”, and you’ll use the same skill on your own product. One made-up example runs through the lesson: an online furniture store’s app.

The lesson, in seven parts

1. 1Set up the critique: purpose, business, and scope.

   Don’t open the app yet. First, agree on what you’re judging it against. An app that sells sofas and an app that tracks your runs need different things from the same screen.

   Say out loud what the app is for and how the company makes money from it. If the interviewer chose the app, ask what to focus on; if you chose it, say why. Then narrow the scope. Twenty minutes is enough for one user and one task, done well. Not the whole app.

   Here are good questions to open with, and habits to avoid:

   Ask

   • “Is there a goal you’d like me to focus on, such as new customers or repeat purchases?”
   • “Should I look at the phone app, or the website?”
   • “Here’s how I think the app makes money. Correct me if that’s wrong.”

   Avoid

   • Opening the app and listing whatever you notice first
   • Trying to cover every screen in twenty minutes
   • Skipping the business: a critique that ignores how the app pays for itself misses half the job

   The template

   [App] helps [user] [get a job done], and the business makes money by [how]. Today I’ll focus on [one user] doing [one task] on [one device].

   Filled in, for the made-up furniture store

   The furniture store’s app helps people furnish a room without visiting a showroom, and the business makes money on each order and on paid delivery. Today I’ll focus on a first-time buyer choosing and buying a sofa on their phone. (Made-up example.)

2. 2Choose one user, and the job they’re trying to get done.

   Every judgment you make depends on who’s using the app. A button that’s “too hidden” for a newcomer may be exactly right for someone who uses the app daily. So pick one user, and say who they are.

   Start with the job: what is this person trying to get done, and what would they do if the app didn’t exist? (The Christensen Institute calls this “the progress they’re trying to make”, as defined in the key terms.) For a sofa, the answer might be: drive to a showroom, sit on a few, and measure the doorway with a tape measure. That alternative tells you what the app has to beat.

   In an interview you won’t have research, so your user is an educated guess. Say so. A short persona keeps you honest:

   Persona

   Sam, a first-time sofa buyer (made-up)

   Job to be done

   Get a sofa that fits the living room and the doorway, without a wasted weekend

   Situation

   Just moved into an apartment; shopping on a phone in the evening

   Worries

   Will it fit? Will it look like the photo? When will it arrive, and who carries it up?

   Familiarity

   New to the app; has bought small things online, never furniture

   Lesson 2 showed that a real persona is built from research. In an interview, a persona is a stated assumption. Say which parts you’d check with real customers.

   Then think about people your persona leaves out. The W3C’s Web Accessibility Initiative groups disabilities that affect how people use digital products as auditory, cognitive and learning, physical, speech, and visual. Asking “could someone with low vision, or with one hand free on a moving train, finish this task?” can turn up problems many people have, only worse. As the W3C puts it, technology designed for a broad range of abilities benefits everyone.

3. 3Walk the core journey, twice.

   Now open the app. Follow your user through one task from start to finish, and narrate what they’re trying to do at each step, not just what you see.

   Nielsen Norman Group’s guide to heuristic evaluation suggests a useful habit: if you don’t know the product, go through the task once just to learn it, without judging anything. Then go through a second time and look for problems. The same guide recommends narrowing the scope to one task, one section, one user group, or one device. That’s exactly the scope you set in part 1.

   For the furniture store, the core journey has three stops: the product page, the room planner, and checkout. Mark each problem where it happens. A problem on the product page loses people before they ever see the planner, so where a problem sits matters as much as what it is.

4. 4Test what you see against clear principles.

   “I don’t like it” is an opinion. “The shopper has to remember the sofa’s width from another screen” is a finding, because anyone can check it. Principles turn your reactions into findings.

   The best-known set is Jakob Nielsen’s 10 usability heuristics. He first developed heuristics with Rolf Molich in 1990, and refined them into this list in 1994. Nielsen Norman Group calls them heuristics “because they are broad rules of thumb and not specific usability guidelines.” Here they are, with what to look for in a shopping app:

   Heuristic	In short	What to look for in a shopping app
   1. Visibility of system status	Keep people informed about what’s going on, with feedback in a reasonable time	Is it clear the item is in the cart, the order went through, and when it will arrive?
   2. Match between the system and the real world	Speak the users’ language, not internal jargon	Do delivery options use words shoppers use, or the warehouse’s terms?
   3. User control and freedom	Give a clearly marked “emergency exit” from mistakes	Can you undo, go back, or remove an item without starting over?
   4. Consistency and standards	Same words and actions mean the same thing; follow conventions	Are sizes always in the same units? Is the cart where people expect it?
   5. Error prevention	Prevent problems before they happen	Does it warn you before you buy a sofa that won’t fit through your door?
   6. Recognition rather than recall	Keep needed information visible, so people don’t have to remember it	Are dimensions visible where you decide, or hidden on another tab?
   7. Flexibility and efficiency of use	Shortcuts for experienced users; let people tailor frequent actions	Can a returning customer reorder, or reuse saved room sizes?
   8. Aesthetic and minimalist design	Leave out information that is irrelevant or rarely needed	Do banners and pop-ups compete with the price and the Add to cart button?
   9. Help users recognize, diagnose, and recover from errors	Plain-language error messages that state the problem and suggest a fix	Does a failed address say what’s wrong and how to fix it, or show a code?
   10. Help and documentation	Easy-to-search help, in context, focused on the task	Is help about delivery or assembly there at the moment you need it?

   Heuristic names from Jakob Nielsen, “10 Usability Heuristics for User Interface Design”, Nielsen Norman Group. The “In short” column paraphrases its descriptions.

   Two cautions from Nielsen Norman Group’s guide to heuristic evaluation keep you honest. Breaking a heuristic isn’t automatically a problem: the guide’s example is a hamburger menu (the three-line icon that opens a hidden menu), which hides navigation (against recognition rather than recall) but is often a necessary trade-off on a small phone screen. And a heuristic evaluation “cannot replace user research.” The guide recommends three to five people evaluating independently, because each one misses things. In an interview you’re one evaluator, so present findings as likely problems you’d confirm with customers.

5. 5Rank the problems by severity.

   By now you might have ten problems. You can’t fix ten. A product manager’s value is in choosing.

   Jakob Nielsen’s article on severity ratings says the severity of a usability problem combines three factors: frequency (is it common or rare?), impact (will it be easy or difficult for users to overcome?), and persistence (is it a one-time problem, or will it bother people again and again?). He adds a fourth consideration, market impact, because some problems hurt a product’s popularity even when they’re easy to overcome. Then he gives a single scale:

   Rating	Nielsen’s wording
   0	I don’t agree that this is a usability problem at all
   1	Cosmetic problem only: need not be fixed unless extra time is available on project
   2	Minor usability problem: fixing this should be given low priority
   3	Major usability problem: important to fix, so should be given high priority
   4	Usability catastrophe: imperative to fix this before product can be released

   From Jakob Nielsen, “Severity Ratings for Usability Problems”, Nielsen Norman Group, 1994. He also warns that ratings from a single evaluator are “too unreliable to be trusted”; the mean of three evaluators is satisfactory for many practical purposes.

   In an interview, rate out loud and say what would change your mind: “I’d call the hidden delivery date a 3, because every buyer meets it at the moment they decide. I’d want to see how many people leave at that step before calling it a 4.”

6. 6Propose improvements, with their trade-offs.

   Pick your top two or three problems and propose a fix for each. Two well-argued ideas beat fifteen bullet points.

   Every fix costs something: engineering time, a slower page, a more crowded screen, or a risk to another goal. Naming the cost before anyone asks shows you’re thinking like the person who has to decide. It also shows you know a fix can make things worse.

   A simple shape for each proposal:

   The template

   The problem: [what the user meets, and where]. Why it matters: [the heuristic it breaks, and its severity]. My idea: [the change]. The trade-off: [what it costs or risks]. How I’d test it: [the smallest way to learn if it works].

   Filled in, for the made-up furniture store

   The problem: Sam only sees the delivery date after entering payment details. Why it matters: it breaks visibility of system status, and I’d rate it a 3, since every buyer meets it at the moment they decide. My idea: show an estimated delivery window on the product page and in the cart. The trade-off: it needs live data from the delivery team, and a wrong estimate could hurt trust more than no estimate. How I’d test it: show the window to a random half of shoppers and compare (an A/B test, lesson 8). (Made-up example.)

   Notice what the fix isn’t. It isn’t “redesign checkout.” It’s one change, tied to one problem, with an honest cost.

7. 7Say how you’d know it worked.

   A critique that ends with ideas is half done. Finish with how you’d measure each one.

   You met this in lesson 4: Google’s HEART framework, from a 2010 paper by Kerry Rodden, Hilary Hutchinson, and Xin Fu. It names five categories of user-centered metrics: Happiness, Engagement, Adoption, Retention, and Task success. Its Goals, Signals, Metrics process works well for a single fix: state the goal, find the signals that would show success, then turn them into metrics you can track. Here it is for the delivery-date fix:

   Step	For the delivery-date fix (made-up)
   Goal	Shoppers know when their sofa will arrive before they pay
   Signals	Fewer people leaving once they see the delivery date; fewer “when will it arrive?” questions to support
   Metrics	Share of shoppers who view a sofa’s product page and go on to finish the order; delivery-date questions to support per 1,000 orders
   Guardrail	Orders cancelled because the estimate was wrong, so the fix doesn’t trade one problem for another

   HEART and Goals, Signals, Metrics are from Rodden, Hutchinson, and Fu, “Measuring the User Experience on a Large Scale”, CHI 2010. This fix sits in the Task success category.

   Measure from a step before the change. The fix moves the date up to the product page, so if you start counting at checkout or payment, people who now leave earlier simply vanish from your number. Then close in thirty seconds. Restate who you looked at, the two or three biggest problems, the fix you’d try first, and how you’d know.

With AI: AI tools can look at a screenshot and list likely usability problems in seconds. That’s a fair way to check you haven’t missed something obvious. But they don’t know your user, your business, or which problem matters most this quarter, and they can state guesses with confidence. Treat them as one more evaluator. Choosing, and explaining why, is still yours.

APP:
What it's for (the job it does):
How the business makes money from it:
Focus agreed with the interviewer (or my reason for it):

USER:
Who (one person, and how familiar they are with the app):
Their job to be done:
What they'd do if the app didn't exist:
Who else to consider (accessibility):

CORE JOURNEY (one task, one device):
Step 1:
Step 2:
Step 3:

FINDINGS:
Where | What the user meets | Heuristic it breaks | Severity (0-4) and why
1.
2.
3.

TOP FIX:
The problem:
My idea:
The trade-off:
How I'd test it:

HOW I'D KNOW IT WORKED:
Goal:
Signals:
Metrics:
Guardrail metric (so it doesn't break something else):

30-SECOND CLOSE:

Ask AI about this lesson

All lessons