When the sponsor of a platform modernization is asked how they know the new system is better than the old one, the answer is usually anecdotal. Users like it, the screens are cleaner, and a few teams report that it feels faster. Those statements can all be true, and none of them is evidence.
The cause is rarely carelessness. By the time anyone wants to prove the program worked, the old platform is gone or going, and the baseline was never captured in terms that match how the new platform will be measured.
What a Program Has to Demonstrate
A modernization is judged on one question: does the new platform perform better than the old one? Answering it has two requirements. The team needs a measured picture of how work runs today, and it needs to measure the new platform on the same terms later.
Most programs invest heavily in the second requirement and very little in the first. Requirements come from workshops. Success criteria come from the business case. The current state is described and not measured, and the description reflects what the most vocal participants remember.
What the Baseline Must Contain
A useful baseline is more than a single average handle time. It describes the work at several levels, across every person and every application, inside the system of record and outside it. Three elements matter most.
The structure of the work. Which processes exist, how often they run, how much time they take and how easy each is to automate. The screen below is a process inventory from the Pyze demo environment, with processes derived from activity patterns instead of an org chart.
The variation. Averages hide important differences. Two teams with the same average can run entirely different procedures, and a platform that suits one may not suit the other. The baseline needs the variants and not only the mean.
The effort that never reaches a system of record. In one global consumer goods order management engagement, 60% of the working day sat outside the system of record, and 18 priced opportunities came from examining that work. A program scoped from the system’s own logs would not have seen that work, and a new platform built from those logs would not have addressed it.
Four Questions After Go-Live
Once the new platform is live, the same measurement method answers four questions against the original baseline.
- Adoption. Is it used, by whom, and for which work? Continued use of the old path is the earliest sign that a design decision was wrong.
- Productivity. Is it faster? Compare handle time by work type with case mix held constant, instead of averaging across a population that has changed.
- Conformance. Is it used as the design intended? Where it is not, decide whether the design or the behavior needs to change.
- Agent effectiveness. If agents or automation are now in the queue, what did they absorb, what returned to a person, and where did the released capacity go? Capacity that cannot be located was not released.
Each of these needs a baseline to compare against. Without one, a program can report activity, such as users logged in or tickets closed, but not whether anything improved.
The Right Sequence
The order matters more than the tooling. Instrument before development begins, not after the first release. Capture with the same method that will measure the new platform, so a change is a configuration update and not a new study. Keep the measurement running, because the baseline needed in month eighteen is the one that includes everything that changed in between.
This is also where the time-and-motion study falls short. A Tier-1 Canadian retail bank used continuous measurement across 1,100 retail centers in place of manual time-and-motion studies, and identified 50K+ hours of annual productivity from it. A study gives a snapshot that cannot be repeated. A running baseline gives a trend that can be verified.
Where to Go Next
- Read how a first baseline comes together in practice: What an Execution Intelligence Assessment Delivers in Six Weeks.
- Explore Productivity Intelligence and the methodology behind measuring before and after.
- Want to see it on your own process? Book a demo.