The regional sales director had a problem with his dashboard that he couldn’t explain.
Completion rates on the company’s AI roleplay training were at 94%. Empathy scores were trending green. New hires sounded polished on practice calls β good pacing, textbook objection handling, the works.
And yet, lost-deal reviews kept surfacing the same failure mode. A rep β usually a confident, fluent one β would state something about a product that wasn’t true. An outdated spec. A warranty term that changed last quarter. A compatibility claim nobody approved.
The training tool was grading how the reps sounded. It wasn’t checking whether what they said was correct.
This is the story of how they fixed that gap.
The cost of confident wrong answers
This company distributes industrial equipment β pumps, motors, control systems β to manufacturing and facilities customers. Their catalog is complex: hundreds of SKUs, dozens of spec variants per product line, warranty terms that differ by model year, and a pricing sheet that updates every quarter as supplier costs shift.
Their sales team of 120 reps handles both inbound RFQs and outbound account management. The average rep carries 30+ active quotes at any time. The margin for error is thin, and the buyers are technical β plant engineers and procurement leads who will check a spec sheet before they check your rep’s tone.
When we first spoke with the VP of Sales, he shared a number that had been keeping him up at night: in a review of Q4 lost deals, 17% cited inaccurate product information as a contributing reason for not moving forward β and he suspected the real number was higher, since not every buyer explains why they walked.
Nearly one in five deals dying because a rep confidently stated something that wasn’t true.
The uncomfortable part: the reps making these mistakes weren’t the struggling ones. They were the high performers β fluent, quick on their feet, good at building rapport. They just weren’t always right.
The training stack was making it worse
The company had invested heavily in AI roleplay training. Reps practiced with virtual buyers, got scored on empathy and objection handling, and could repeat scenarios as many times as they wanted.
The problem was hiding in plain sight. When a rep practiced a call and confidently stated a 5-year warranty on a product that only carried a 3-year warranty, the platform gave full marks for objection handling. The content of the claim β whether it was true β was never evaluated.
Because the platform rewarded fluency, reps were being trained to sound more certain, not more correct. The training wasn’t just failing to prevent misinformation. It was reinforcing it.
The intervention: claim-level verification
The company piloted a different approach with EOS. Instead of grading the shape of the conversation, EOS verifies the content of every factual claim a rep makes during a practice session or an uploaded real call:
- Claim extraction β every practice call is transcribed and analyzed. Each factual claim (spec, price, warranty term, compatibility statement) is isolated as a checkable assertion.
- Document grounding β each claim is checked against the company’s own current product documentation, not a general-purpose model’s memory of what the product might be.
- Three verdicts β each claim comes back supported, contradicted (directly conflicts with the documentation), or insufficient (too vague to verify, or missing a required qualifier).
Every contradicted claim generates a follow-up quiz question, with a citation back to the source document. The output isn’t a score β it’s a certification trail: which claims were tested, against which document version, on which date, and what the rep got wrong.
The 90-day results
The company ran the pilot with 40 reps over 90 days.
Lost deals attributed to bad product information dropped 73% (17% β 4.6%) β the headline result, and the one tied directly to revenue. Contradicted claims per session also fell sharply, down 71% (2.8 β 0.8).
The more interesting story was qualitative. Reps who assumed they “knew the catalog” discovered gaps they didn’t know existed. The auto-generated quizzes β built from each rep’s actual mistakes β turned an abstract “go study the spec sheet” into a concrete “here’s exactly what you got wrong, and here’s the document that proves it.” Several reps who had been with the company for years were flagged on specs they’d been misstating for a long time β errors nobody had caught because nobody had been checking.
Is your training stack checking facts β or just style? Our 12-page report, Beyond Roleplay: The Rise of the Sales Knowledge Engine, includes the full evaluation framework, the 10-question stack audit, and the data-sovereignty questions to ask any vendor. [Download the whitepaper β]
Why this worked
Traditional sales training treats “product knowledge” as static β learn it once, you’re good. In a catalog that updates quarterly, product knowledge isn’t static. It’s a moving target.
A rep who aced a product-knowledge test six months ago might be confidently stating outdated information today β and neither they nor their manager would know until a deal dies. What this company built with EOS wasn’t just better training. It was a continuous verification loop: reps practice β claims get checked against current documentation β contradictions generate targeted quizzes β reps close the gap β repeat. The system doesn’t assume knowledge. It proves it, using the manager dashboard to show exactly who’s certified against which current document.
The takeaway for sales leaders
This story isn’t unique to industrial distribution. Any organization selling a complex, changing catalog β telecom, insurance, automotive, financial services, electronics, industrial equipment β faces the same gap.
Your reps are getting smoother. Your training tools are getting better at grading how they sound. But if nobody is checking what they actually say against your current documentation, you’re flying blind.
The question isn’t whether your reps are improving. It’s whether they’re improving at saying the right things β or just at sounding confident while saying the wrong ones.
See your own docs become a knowledge check. EOS turns your product documentation into practice and provable knowledge β claim extraction, fact-grounded verification, and auto-generated quizzes that certify what reps actually know. Start free with up to 5 seats at akaeos.com, or [download the full whitepaper].
Leave a Reply