The Question Nobody Asks Until It’s Too Late…
If that question makes you pause, you already have your answer.
A program can run for twelve fully funded months, hit every activity on schedule, and still land on a funder’s desk with a result nobody saw coming.
The work wasn’t weak; nobody was checking the gap between the plan and the data until the year was already over.
This is the moment that ends a funder relationship. Not the missed target itself, but the fact that nobody saw it coming until the door was already closed.
Let’s audit the logic.
Programs rarely fail overnight. They drift slowly, quarter by quarter, decision by decision, and the drift only becomes visible once someone finally looks at a full year of data at once.
By then, there is no time left to fix it. Only time left to explain it.
The Check Engine Light Problem
A car does not go from fine to broken in a single moment. Long before the engine fails, a warning light comes on the dashboard.
The driver who checks it that week replaces a belt for forty dollars. The driver who ignores it for six months replaces the engine for four thousand.
Program evaluation has the same warning light. It is called variance, the gap between what a KPI was supposed to show at month four and what it actually shows.
The problem is not that programs generate variance. Every program does. The problem is that most organizations only check the dashboard once, at the end, when the engine has already failed, and there is nothing left to repair.
This is what the Diagnose (D) step in OLPADR exists to catch.
Not a year-end audit, but a running comparison between the plan and the data is checked on a cadence, so a variance gets flagged in month three instead of being discovered in month twelve.
What an Audit Actually Catches
An evaluator-grade audit is not looking for effort. It is looking for three specific failure types, and each one has a different fix.
→ Design failure: The plan itself was built on a flawed assumption. The activities were completed exactly as written, but the logic connecting those activities to the intended outcome was wrong from day one.
→ Fidelity failure: The plan was sound, but execution drifted from it. Staff adapted the program on the ground without updating the model, so the thing being measured is no longer the thing that was actually delivered.
→ Logic failure: The activities happened, engagement is real, satisfaction scores look good, but the outcome itself never moved. Something in the causal chain between action and result simply does not hold.
Three different diseases and three different treatments. A year-end report tells you the patient is sick. A running diagnostic tells you which organ to treat, and does it while treatment is still possible.
Run This Mini Self-Audit Right Now
Before booking anything, pull up your current program dashboard and answer these four questions honestly.
→ Do you have a documented baseline for every KPI, captured before the program started, or are you comparing current numbers to a rough memory of where things stood?
→ Is there a written definition for each indicator, specifying the numerator, denominator, and data source, or does everyone on the team have a slightly different idea of what the number actually means?
→ Who owns checking these numbers against the plan, by name, and on what schedule? If the honest answer is “whenever someone remembers,” that is your answer.
→ When was the last time a number moved in the wrong direction and someone acted on it within thirty days, rather than at the next scheduled report?
If two or more of these came back weak, the issue is not your program. It is your governance around the program, and that is fixable in weeks, not quarters.
The Repair Sequence: Baseline, Cadence, Owner
Once a gap is found, the fix follows the same three-part sequence every time.
→ Baseline: lock the starting number for every indicator before anything else moves. Without this, “improvement” is just a feeling with a percentage attached to it.
→ Cadence: set a fixed interval, weekly or monthly, depending on the program, where the current number gets checked against the plan. Not reviewed. Checked, with a clear threshold for what counts as a variance worth flagging.
→ Owner: name the person responsible for that check. A number with no owner does not get monitored. It gets discovered, usually too late.
This is the same architecture behind our OLPADR Evidence Ledger and Program Evaluation services, built so that leadership can see variance the month it appears, not the year it becomes unfixable.
Governance That Behaves Like an Evaluator
The organizations that survive scrutiny are not the ones with perfect programs.
They are the ones whose governance behaves like an evaluator year-round, checking baselines, defining terms precisely, assigning ownership, and treating a variance as information rather than an accusation.
That is what an Evaluator-in-Residence changes. Not the work itself. The moment someone finds out whether the work is on track.
If we audited your data today, what would we find first: missing baselines or inconsistent definitions?
Book an Evidence Ledger Audit, a 3-week engagement that finds the gaps in your current measurement system before your next review cycle does it for you.
Evidence-led, impact-driven. The OLPADR way.