The VP of Sales had a simple question: could he trust his training platform?
His team sold industrial automation equipment — PLCs, drives, sensors, and control systems — to factories and OEMs. The catalog was enormous. Specs changed with every firmware release. Compatibility matrices shifted. A wrong claim about a controller’s I/O count or a drive’s power rating could mean a failed installation, a stalled production line, and a customer who never called again.
The training platform he’d bought two years earlier showed strong completion rates. But he had never actually tested whether it would catch a wrong spec. He’d assumed it would.
Then he read a blog post that described a method he could run himself, without a vendor demo or an NDA. It was called the public-manual test.
He decided to run it on his own platform.
The setup: why he tested his own stack
The VP of Sales had been through enough vendor demos to be skeptical. Every platform looked good in a controlled environment. Every vendor claimed their system checked facts.
But he’d never actually verified it on his own terms, with material the vendor hadn’t prepared for.
The public-manual test was designed exactly for that. It used only public information — product manuals anyone could download — and it worked on any platform, including the one he was already paying for.
Following the test’s own guidance to spread across categories rather than stay inside one industry — so a strong or weak result couldn’t be waved away as “his industry’s manuals happen to be easy” — he picked five public manuals from different product categories: a consumer electronics device, a car’s trim-and-options comparison page, a telecom carrier’s plan and device page, an industrial sensor, and a variable frequency drive. Only the last two were from his own world; the other three were products he had no professional stake in either way. All five had one clear, checkable spec he could verify.
The method: stating the wrong spec confidently
The test procedure was straightforward.
For each manual, he found one clean spec with a single correct answer — a power rating, a response time, a warranty length, a data allowance, a trim’s feature inclusion.
Then he uploaded the manual to his current platform, started a practice roleplay, and stated the wrong version of that spec the way a fluent, confident rep would say it. No hedging. No “I think.” No “let me check.” Just the kind of confident statement that closes deals when it’s right and kills them when it’s wrong.
He ran each test two to three times per manual to check whether the result was a pattern or a fluke.
Before he started, he defined “pass” — and “fail.”
A fail meant the platform’s feedback flagged the claim as wrong and, ideally, cited the document it was checking against.
A pass meant the platform’s feedback said nothing about the claim at all. It graded tone, pacing, and empathy — and never mentioned that the spec he’d stated was made up.
The result: silence, five times over
Every single test came back the same way, regardless of the product category.
The platform scored his pacing. It complimented his empathy. It noted his objection handling was strong. And it said nothing — absolutely nothing — about the fact that he’d confidently stated a wrong spec, whether that spec was a drive’s power rating or a phone’s battery life.
Five manuals, five different industries. Five wrong claims. Zero flags.
The silence wasn’t a neutral outcome. It was the result. The platform had graded how he sounded and had no mechanism to check what he actually said.
“I wasn’t surprised that it didn’t catch every error. I was surprised that it didn’t catch any — and that it didn’t matter which industry the manual came from. Five different products, five different spec types, and the feedback never mentioned a single one. It wasn’t checking. It was never checking.”
Have you run the public-manual test on your own platform? Our 12-page report, Beyond Roleplay: The Rise of the Sales Knowledge Engine, covers the full verification architecture behind this test, the questions to ask any vendor, and how to evaluate what you already have before you renew. [Download the whitepaper →]
Why the silence mattered for his business
The VP of Sales took the results to his sales enablement lead. She wasn’t surprised either.
She had been tracking post-sale issues for two quarters. In that time:
- 13% of installation support tickets involved customers who had been given incorrect technical specifications during the sales process
- 9% of compatibility issues — equipment that didn’t work with the customer’s existing systems as promised — traced back to inaccurate claims made on calls
- New technical sales reps were taking an average of 11 weeks to reach a level where they could run customer conversations without a senior engineer present
The training platform had been generating reports. It hadn’t been preventing errors.
“We were measuring how our reps sounded. We had no idea whether they were right.”
The intervention: a platform that passes the test
He ran the same public-manual test — the same five manuals, the same cross-category mix — on EOS (app.akaeos.com).
This time, the results were different.
For each wrong spec he stated, EOS extracted the claim, checked it against the uploaded manual, and returned a specific verdict:
“Incorrect: The drive’s rated output is 7.5 kW, not 5.5 kW. See Manual Section 3.2, Power Ratings.”
“Incorrect: The sensor’s response time is 2 ms, not 5 ms. See Datasheet, Operating Specifications.”
“Partially correct: The controller supports 16 digital inputs, not 32. See I/O Configuration Table, p. 14.”
Every verdict came with a citation. Every citation pointed to the exact passage in the document he’d uploaded — regardless of whether that document was an industrial datasheet or a consumer electronics spec sheet.
The difference wasn’t subtle. One platform graded confidence. The other verified facts.
Why the public-manual test works
The VP of Sales didn’t need a vendor demo to learn what his old platform couldn’t do. He didn’t need a trial account or a sales call. He needed an afternoon and five public manuals from outside his own industry.
The test worked because it removed every advantage a vendor could have. No platform had seen these manuals before. No script had been optimized to make any platform look good. The material was public, neutral, and picked by him — not the vendor. Spreading the five manuals across categories, instead of staying inside his own industry, meant the result couldn’t be explained away as “industrial specs happen to be easy to verify.”
The silence he found on his old platform wasn’t a bug in the demo. It was a structural gap. The platform had no mechanism to verify claims, so it never verified them — regardless of how many manuals he uploaded, or which industry they came from.
The platform that passed the test didn’t have that gap. It extracted claims, checked them against the source, and returned a verdict, on a phone’s spec sheet just as reliably as on a drive’s datasheet.
“I ran the test in an afternoon. It told me more about my platform than two years of dashboards ever did. The silence was the answer.”
The takeaway for sales leaders
If you’re evaluating an AI sales training platform — or wondering whether the one you already have is actually doing what it promised — you don’t need a vendor demo to find out.
You need five public manuals spread across a few different industries, a practice session, and one confidently stated wrong spec per manual.
If the platform says nothing, you’ve found the gap. It’s not a hypothetical. You just watched it happen on your own platform, with material you picked yourself, in about an afternoon.
The public-manual test is the closest thing to a fair, repeatable comparison you can run yourself, on your own terms, before you sign anything — or before you renew what you already have.
And for an industrial equipment manufacturer selling controllers, drives, and sensors where a wrong spec means a failed installation and a lost customer, that test isn’t optional. It’s the first thing you should run.
Run the test. Read the silence. EOS passes the public-manual test — claim extraction, fact-grounded verification, and citation-backed verdicts for every spec a rep states, regardless of industry. Start free with up to 5 seats at app.akaeos.com, or [download the full whitepaper].
Leave a Reply