Product Manager Capstone Lab
Build a capstone around the AI release you would send back
Give a reviewer one decision they can challenge and the evidence that would change your mind.
A portfolio deck says an assistant achieved 88% accuracy. A reviewer asks which failures you are prepared to ship. The candidate who can open three cases and explain their release decision has a stronger capstone than the one with another chart.
Imagine a support assistant helping customers change delivery dates. Create twelve fictional requests with expected outcomes. Include straightforward changes, dates past the cutoff, and a request whose address matches two orders. Mark the cases as invented throughout. They demonstrate your reasoning without suggesting that you had access to customer data.
Define what counts before running the cases
An ordinary change passes when the correct order shows the requested date. An ambiguous case passes when the assistant asks which order the customer means and leaves both orders alone. Anthropic’s evaluation guide describes checking the resulting state of a task alongside the conversation. That distinction matters here because a friendly answer can accompany a change to the wrong order. Anthropic • Evaluating agent outcomes
Write a release memo with one hard condition. For instance, every ambiguous-order case must preserve both original orders until the customer clarifies. Show the request, the before-and-after state, and your judgment for one passing case and one failing case. Include the person who would own the fix.
End your presentation by handing the reviewer a thirteenth request you have never tested. Ask how they would score it under your published criteria. If two reasonable readers disagree, you have found a gap in the specification worth discussing in an interview.