Extraction beats summarisation
Unit 6 of 8 · Lesson 1 of 3Index
Wednesday. Priya exports everything: four thousand support tickets, eight hundred reviews and sixty sales call notes, from three different tools. Nobody at Sona has ever read all of it.
A summary throws the evidence away
The tidy summary
Maya pastes it into a chatbot and gets “users want better onboarding”. It sounds right, and you cannot check a single word of it.
One row for every message
Where it came from, when, the exact words, and one label. Now every number leads back to a person.
- Three tools, nobody reading01
Three tools, nobody reading
Tickets in the help desk, reviews in the store, call notes in a doc. All of it skimmed. None of it counted.
- Fix the labels first04
Fix the labels first
Write the list before you run anything. Anything that does not fit goes to Other, never to a label the model just made up.
- Read the Other pile05
Read the Other pile
It is small and messy. It is also where the thing nobody expected is hiding.
Follow a claim back to its source
Interactive teaching example · All numbers and tickets are fictional.
“I retried because nothing changed.” Fictional ticket A
- Observed action
- Retried the operation
- Interpretation, not a fact
- Possible missing feedback
Tickets A and B may share a feedback issue. Ticket C does not support that claim. Three selected tickets cannot establish how common the issue is across the product.
How Amazon handled it
Doing it with AI, and where it breaks
Write one extraction prompt with a fixed schema and run it in batches over the whole export. Load the rows into a sheet and pivot.
Around row three hundred, models drift and start inventing labels. Tell it to use only your list and send everything else to OTHER, then read every OTHER yourself.
Return one JSON object per item with keys: source, source_id, date, verbatim_quote (copied exactly, never paraphrased), label (exactly one of: BILLING, ONBOARDING, SYNC, ACCURACY, PERFORMANCE, TRUST, OTHER), severity_1_to_5, segment_guess. Never invent a label outside the list. If nothing fits, use OTHER.
Build it
An extraction pipeline and a coded signal log of at least 200 rows, each with its exact quote.
Claude or ChatGPTGoogle Sheets
The worked solution
Stuck, or want to compare?
There is a worked solution. Try the build first. Reading it before you have attempted anything is the fastest way to learn nothing.
Checkpoint
If you did the build, these take two minutes. If you cannot answer one of them, that is the part to go back to.
Moving on marks this lesson complete. Finish the build first, it is the part that counts.