You’re in the budget meeting. Someone asks whether the training program is actually working.
“The team completed 1,200 roleplays this quarter” is a true sentence. It is also not an answer. It tells the room that training happened. It tells them nothing about whether anyone learned anything, whether the gaps are shrinking, or whether the money should keep flowing next quarter.
That’s the gap this post is about — not because participation metrics are meaningless, but because they answer a different question than the one a budget conversation is actually asking.
Why participation metrics don’t survive the room
Hours completed, course completion rate, empathy or tone scores — these are the numbers most training dashboards lead with, and they’re not wrong to track. They tell you people showed up and did the work.
What they can’t do is distinguish a team that’s actually getting more accurate from a team that’s just getting more practice. A rep can complete every assigned roleplay and still be confidently wrong about the same product detail in week twelve that they were wrong about in week one. The completion number goes up either way.
A CFO — or anyone else holding the budget pen — has heard “we did a lot of training” before. It’s not a sentence that gets a budget renewed on its own merits.
What actually holds up in that room
The alternative isn’t a fancier participation number. It’s a shift in what gets measured at all — from “did training happen” to “is the team’s accuracy actually moving.”
A few things worth building your reporting around:
A pass rate, tracked over time. Rather than reporting hours logged, report the share of scored evaluations that clear a real bar — say, 80 or above on the criteria that matter to you — for a defined window, and how that compares to the window before it. “72% of scored evaluations hit our bar in the last 30 days, up from 61% in the prior 30” is a sentence with a trend line in it. “We did more roleplay” is not.
A defined comparison window, not a vague trend. Pick a period — 7, 30, or 90 days — and always compare it to the equivalent period immediately before it. Consistency here matters more than the specific window you choose; a number that moves against a shifting baseline isn’t a trend, it’s noise.
A short list of who needs attention, not a ranking of who’s best. The most useful signal for a budget conversation usually isn’t “here’s who improved the most” — it’s “here’s who’s going quiet, whose scores are slipping, or whose quiz performance dropped, and here’s what we’re doing about it before it becomes a bigger problem.” That’s a leading indicator a budget holder can actually act on.
See the full framework behind this shift. Our 12-page report, Beyond Roleplay: The Rise of the Sales Knowledge Engine, covers the move from participation metrics to verification metrics in more depth, along with the architecture that makes accurate tracking possible in the first place. [Download the whitepaper →]
What this sounds like in the room
Compare these two statements:
“Our team completed 400 hours of roleplay training this quarter.”
“In the last 30 days, 72% of scored evaluations hit our 80-point bar, up from 61% the prior 30 days. Four reps are currently flagged for attention — down from nine last month — and we’re addressing each one individually.”
The first sentence describes effort. The second describes a trend, a bar, and a plan. Only one of those is something a budget holder can actually evaluate.
What not to promise yet
Be careful not to oversell what this kind of reporting can currently do, because overselling it undermines the credibility this whole approach is trying to build.
Most platforms in this category — including the more mature ones — don’t yet offer a one-click, CFO-ready export of these numbers. Be prepared to walk through a live dashboard rather than hand over a polished PDF. Most also don’t rank reps by “most improved,” and most don’t break down, at the team level, which specific evaluation criterion is failing most often — that level of detail usually lives inside an individual session report, not a team-wide rollup. Say what your reporting can actually show, and don’t promise a dashboard export or a leaderboard that doesn’t exist yet. A budget conversation survives an honest “here’s what we can show you live” much better than it survives a promised report that turns out not to exist when someone actually asks for it.
The takeaway
Retiring “we did a lot of training” isn’t about finding a more impressive-sounding number. It’s about reporting something a budget holder can actually act on — a trend that’s moving in a real direction, against a consistent comparison window, with a short list of exactly who needs help next. That’s a genuinely different conversation than “here’s how many hours we logged,” and it’s the one that actually gets budgets renewed.
See your own docs become a knowledge check. EOS turns your product documentation into practice and provable knowledge — claim extraction, fact-grounded verification, and auto-generated quizzes that reveal what reps actually know. Start free with up to 5 seats at app.akaeos.com, or [download the full whitepaper].
Leave a Reply