AI Product Management
Ship an AI feature you can defend: a versioned prompt, an eval suite that turns “feels better” into a number, a cost per interaction, and an interface designed for the times the model is wrong.
Sona wants an assistant that answers money questions from a user's own transactions. Maya has already told the board it ships this quarter. This is where AI stops being a tool you use and starts being the thing you are accountable for when it tells someone the wrong balance.
- 7
- chapters
- 24
- lessons
- 24
- things you build
- ~23h
- of material
You are a competent PM and the AI conversations at your company are happening without you.
What you will be able to do.
Describe your feature's model path well enough that an engineer signs off on it
Decide honestly whether this needs a model at all, and write down what the non-AI version would have cost
Build an eval suite that turns “feels better” into a number, and gate every prompt change on it
Design retrieval, then find the query where it answers confidently from the wrong document
Cost and time a single interaction, and design the interface for the eight percent where the model is wrong
Red team your own feature and write the report before someone on the internet writes it for you
7 chapters, 24 lessons.
Lessons open in order. You can go back to anything you have already reached, and forward one step at a time.
Finish level 04 first.
This level assumes the work in the one before it. Nothing here is hidden behind a payment, only behind the level below.
Go to level 04The working AI feature
A functioning AI feature with a versioned prompt in a repo, an eval suite of at least twenty golden cases scored per version, a retrieval design with a documented failure case, a cost and latency model per interaction, and a red team report with the top three mitigations shipped. Judged the way a real launch review judges it, which is whether you can answer what happens when it is wrong.
- A live AI feature with a versioned system prompt and a changelog carrying eval scores
- An eval suite of 20 or more golden cases, scored across at least three prompt versions
- A cost and latency sheet: p50 and p95 per interaction, and the unit economics of one button press
- A red team report: attacks attempted, what got through, severity ranking, mitigations shipped
Finishing every lesson unlocks Level 06 straight away, free. The AI Product Builder badge itself is reviewed by a practising product manager and is part of the one time community fee.
Submit your capstone
Share your project, supporting evidence, and published write-ups. Your level is included automatically so the reviewer has the right brief.