Share:

Have you ever designed a new performance measure – a powerful one you felt excited about – and then discovered the data you needed was not in the right format or detail? Of course you have! It’s a common challenge when you decide to measure something more deliberately or more meaningfully than you have in the past. But don’t let the data problems put an end to your powerful new measure!

We won’t ever get the data we need, or the measures that matter, if we let our existing data constrain our choice of measures. Sometimes we can immediately repair the data situation, but often we have to wait until the time is right. In the meantime, we can use proxy measures to stand in for our ideal measures.

Illustrating this point, as part of their suite of strategic performance measures, a municipality designed a new measure to support their goal of reducing their environmental impact. The measure they wanted was Electricity Consumption, the total kilowatt hours of electricity they consume through their operations, by month.

But when they attempted to source the data for Electricity Consumption, they could not get any raw data about their actual monthly consumption. The only data they could get (from their electricity provider) was electricity meter readings. And to make it a bigger challenge, these readings were taken at varying locations, at varying points of time throughout each month, and at varying intervals ranging from less than a month through to approximately quarterly. Messy!

They were ready to give up. And the reason they felt like giving up was that they were fixated on the notion that performance measure values have to be accurate. That notion is a fallacy. The power of performance measures is in the patterns of how their values change over time, not in the actual individual values themselves. In PuMP we have a mantra for this: “patterns, not points!”

So the data from the irregular meter readings would be highly correlated over time with electricity consumption. And so it was used to compute a proxy measure of Electricity Meter Readings. And particularly when we use XmR charts to help us focus on the patterns, not the points – or the signals, not the noise – we can still draw conclusions about changes in Electricity Consumption, based on the proxy measure of Electricity Meter Readings. Trends over time are more accurate than the points that make them up.

That’s because the errors in the proxy measure, due to the irregularity of meter readings, are accounted for in the variability measured by the XmR chart. So signals of change in Electricity Meter Readings are indicative of change in Electricity Consumption. Proxy measures can be very powerful, even if they lack accuracy.

DISCUSSION:

Do you use any proxy measures? Do you agree it’s far better to have a proxy than no information at all?