The transcript captured the call and lost the callback
The AI transcript looks complete: greeting, issue, empathy, wrap-up. The customer said call me tomorrow after three. That sentence never became a task.
Post-mortem of a familiar failure: AI call transcripts and summaries that sound finished while dropping the callback promise. Treat promised callbacks as required extractions into the CRM with owner, window and phone, not as optional prose. Review one call against its transcript weekly until callbacks stop disappearing.
What happened
Tuesday, 4:18 p.m. A customer reaches the AI receptionist about a delivery that missed its window. She is calm. She asks someone to call her tomorrow after 3:00 p.m. at the number on the account, or on her mobile if that fails. The assistant agrees. The call ends pleasantly.
Wednesday at 3:40 she has not been called. She dials back angry. The human agent opens the ticket. The transcript reads like a finished conversation: apology, tracking check, empathy language, polite goodbye. The summary says "customer inquired about delivery; reassured." There is no callback task. There is no time window. There is no second number.
This is not the same failure as a summary that drops a renewal condition in prose—we covered that pattern in what the AI summary drops. Here the missing object is an operational commitment: a future action with a clock.
Sequence of work
Where the callback disappeared
- Customer asks
Call tomorrow after three.
- Assistant agrees
Spoken commitment on the call.
- Transcript
Words may be present in the long text.
- Summary
Commitment shortened away.
- CRM
No task, no owner, no window.
What we believed
We believed a "good" transcript was a faithful record of words. We believed a short summary helped agents move faster. We believed the model would "obviously" surface action items because demos always showed neat bullet lists.
We also believed the CRM would be updated somehow—by the model, by a Zap, by the next human who read the thread. Somehow is not an integration.
A transcript can be accurate word-for-word and still miss the only commitment that mattered.
What was actually true
The transcript can be nearly word-perfect and still fail the job. Callback promises are easy to bury in conversational filler: after the empathy paragraph, before the goodbye. Summarizers trained to be concise cut commitments that sound soft ("maybe give me a ring tomorrow afternoon") even when the assistant already agreed.
There was no required schema. Nothing in the pipeline refused to close a call without callback_at, callback_number, callback_owner. Optional bullets in a summary are not a control. Optional bullets vanish under load.
Nobody owned weekly sampling. The quality dashboard tracked average handle time and containment rate. It did not track "promised callback created as a task." You get what you count.
What to change
Make callback a required extract. If the assistant agrees to a future contact, the system must write a task with time window, destination number, and owner queue before the call is marked done. If extraction fails, the call routes to a human with the audio highlighted—not to a neat empty summary.
Separate summary from commitments. Keep the prose summary if you want. Put commitments in structured fields agents sort by. Mixing them is how commitments become adjectives.
Sample against audio, not against vibes. Once a week, listen to five contained calls where the customer said "call me" or "follow up." Compare audio, transcript, summary and CRM. Publish the miss count where the vendor can see it.
Stop buying containment that cannot write tasks. A receptionist that closes the loop in conversation but not in software is theater. Prefer tools that emit structured actions your CRM already understands.
If you still need a human handback packet when the agent is stuck, keep that path too—stuck agents need packets—but callbacks should not require a stuck state to be recorded.
Record the miss count in the same place you record model version, so prompt changes cannot claim victory while the ledger still leaks commitments.
Decision guide
Prose summary vs required commitment
| Consideration | Optional summary bullets | Required callback fields |
|---|---|---|
| Close the call | Allowed with empty bullets. | Blocked until fields or human handoff. |
| Agent next day | Reads a pleasant story. | Sees a due task with a number. |
| Quality metric | Containment and handle time. | Missed-commitment sample rate. |
| Customer | Waits; calls back angry. | Gets the call in the window. |
How the miss usually hides
Containment dashboards celebrate calls that never reached a human. Callback misses hide inside that celebration. The customer was "contained" on Tuesday and became a furious inbound on Wednesday, counted as a new contact with a different reason code. Unless you join those two contacts, the model looks successful while trust falls.
Also watch multilingual calls and noisy mobile audio. Extraction that works on clean headset English may drop commitments on a speakerphone in a second language. Your sample set should include those calls on purpose, not only the demo-quality ones vendors play.
Finally, beware soft agreements. "Someone will reach out" without a time is already a weak commitment; models love summarizing it as reassurance. Train assistants to confirm a window—"tomorrow after three at this number"—so extraction has a chance, and refuse to close without that window when a callback was requested.
A transcript can be accurate word-for-word and still miss the only commitment that mattered.
Put this into practice
If legal wants shorter audio retention, keep commitment fields longer than audio—the ledger is the durable artifact customers feel. Write it down where peak hires will actually read it. Write it down where peak hires will actually read it. Write it down where peak hires will actually read it.
One last check: ensure after-hours callbacks land on a queue that is actually staffed in that window. A perfect CRM task assigned to an empty night queue is how you recreate the miss with better paperwork.
When you renegotiate the vendor contract, add a clause about structured commitment export—not only "AI quality improvements." Quality without a schema is a slide. Schema with a miss-rate report is an operating agreement.
Keep audio retention long enough to support the weekly sample. If audio disappears in forty-eight hours, your post-mortems will argue from memory, and memory always sides with the dashboard.
Wire the required fields before you argue about model vendors. A weaker model that must emit callback_at will beat a prettier summarizer that may omit it. Once the ledger exists, you can compare models on miss rate honestly.
Tell frontline humans what changed. If they still type callbacks into free-text notes out of habit, the structured field will rot. A week of floor coaching usually matters more than another prompt adjective.
If your AI receptionist vendor cannot write tasks to your CRM, treat that as a buy/build decision, not a temporary gap. Temporary gaps around commitments become permanent customer pain.
Today, search last week's transcripts for "call me," "call you," "tomorrow," and "after." Open the matching CRM records. Every hit without a task is a post-mortem of your own. Fix the schema before you tune the prompt. Prompts do not enforce ledgers.
We build AI applications and agents that keep commitments as data. If your transcripts sound complete and your customers still wait by the phone, talk to us.
Related work: Spin:Market.
We build alongside the people doing the work.
Bring us the problem you’re thinking about.