A board deck can contain all the right numbers and still leave your organization exposed.
The problem is not always the metric itself.
Sometimes it is the logic behind the metric.
A percentage may look precise. A trend may look convincing. A KPI may appear ready for executive decision-making.
But if you cannot explain where the number came from, who it represents, what period it covers, and whether the underlying records support the reported result, the number may not hold up under scrutiny.
The ICT Training evaluation provides two useful examples.
One involved the absence of a unique student ID linking pre- and post-results.
The other involved an attendance figure that did not reconcile with the broader source log.
The report presented 95% attendance.
The source log, when reconciled, showed 82.1%.
That is a 13-point difference.
These are not simply technical data issues.
They affect what leadership can confidently say about program performance.
Why your board needs more than a polished metric
Executives make decisions using the information placed in front of them.
If a board receives a metric showing 95% attendance, it may reasonably assume that the number represents the relevant reporting period and that the underlying records support it.
But what if the 95% figure came from a narrower snapshot?
What if the full-period source log tells a different story?
The issue is not that the original number necessarily had no value.
The issue is whether the context surrounding the number was preserved.
A defensible metric needs more than a percentage.
It needs a clear trail back to the evidence.
Data integrity gap #1: No unique student ID
One of the most important structural questions in participant-level evaluation is:
Can you tell whether the same person appears across measurement points?
Consider a program with baseline and follow-up assessments.
Without a unique participant identifier, you may have two datasets containing similar numbers of participants.
But you cannot confidently establish that the individuals in the two datasets are the same.
That limits the claims you can make about individual-level change.
A unique participant ID creates the connection.
It allows the evaluation team to link records across instruments and measurement points.
The result is a much stronger foundation for statements about change.
This is why data architecture matters.
A missing column can create a much larger evidence problem later.
Data integrity gap #2: When a snapshot becomes the headline
The second issue involves reporting period consistency.
A snapshot can be useful.
The problem begins when a snapshot is presented without enough context to distinguish it from a full-period result.
In the ICT Training case, the reported attendance figure was 95%.
When the broader source log was reconciled, the figure was 82.1%.
The difference was approximately 13 percentage points.
That should trigger a diagnostic conversation.
Not:
“Which number looks better?”
But:
“What does each number represent?”
Ask:
- What dates are included?
- Which records are included?
- What denominator was used?
- Was the same calculation applied throughout?
- Does the headline figure represent the full reporting period?
Those questions help prevent a snapshot from becoming an unsupported headline.
Five questions to ask before your next board meeting
You do not need a complicated audit to begin.
Take your five most important metrics and ask:
1. Can we trace the number?
Can someone identify the source record, dataset, or calculation behind it?
2. Can we identify the participants?
If the claim is about participant-level change, can the records be linked?
3. Is the sample consistent?
Are the comparison groups defined the same way?
4. Could a ceiling effect be hiding change?
If most participants already score near the top of a scale, a small numerical movement may not tell the full story.
5. Is this a snapshot or a full-period result?
Make the reporting window explicit.
These questions help leadership distinguish between a number that is simply available and a number that is defensible.
What a stronger board conversation sounds like
Instead of asking only:
“Did we hit the target?”
Ask:
“What evidence supports this number?”
“Can we trace it to the source?”
“Who is represented in the calculation?”
“Does the reporting period match the claim?”
“What did the evidence tell us to change?”
These questions shift the conversation from performance reporting toward evidence-informed decision-making.
That is the real value of evaluation.
Your board deck is not the evidence
A board presentation is a communication tool.
It is not the underlying evidence system.
The slide may contain the final percentage, but the defensibility of that percentage depends on everything underneath it:
Source → Definition → Calculation → Participants → Reporting period → Finding → Decision
If one of those links is weak, the final number deserves a closer look.
Before your next board meeting, audit the five numbers you expect executives to rely on most.
Do not wait for someone in the room to ask:
“Where did this number come from?”
Know the answer before they do.
Book a Diagnostic Call if you want to examine the evidence behind your organization’s most important program metrics: https://www.claritytoimpact.com/diagnostic-call/?utm_source=linkedin&utm_medium=social&utm_campaign=diagnostic_call
Evidence-led. Impact-driven. The OLPADR way.