3 Good Reasons to Avoid Composite Indexes and Scoresby Stacey Barr |
You might think that rolling up your performance measures or KPIs into a weighted score or composite index is a great strategy to simplify your dashboard, to deal with lots of measures, or to create proxies for intangible goals. Think again.
An index (or composite index) is the term we give to a measure that is computed by taking a weighted sum or weighted average of a collection of other measures or scores.
For example, you might take a dozen or so different measures to do with productivity, and create a weighted average of those measures to give you an Overall Productivity Score. Or you might create a Business Unit Performance Index for each of the business units in your company or organisation.
It might sound like a good idea, but it isn’t. Here are three reasons why indexes and scores are a bad idea:
Reason 1: A composite index hides the real problem of too many measures.
If you’re using an index because you have too many measures to digest in one sitting, then the problem is that you have too many measures! Performance measurement is not about measuring everything you can, it’s about measuring only those things you can and should and will do something about.
Suggestion #1: Test the value of each of your performance measures using this checklist of criteria of excellent performance measures.
Reason 2: A composite index hides the real signals of change.
The signal your index gives you will be a diluted average of the signals of all the underlying measures. If you have one underlying measure tracking very well and another underlying measure tracking very poorly, they will average out in the index.
Suggestion #2: Rather than summarising every measure into an index or score, use simple formating to highlight the signals in each underlying measure, to make it easy to focus just on those measures that need attention. Learn how from dashboard guru, Stephen Few, at www.perceptualedge.com.
Reason 3: A composite index hides the tangible results you are trying to monitor.
When you look at an index or score, it’s usually a rolled-up summary of a broad performance area, like productivity or customer service or financial performance. But these broad performance areas aren’t tangible or actionable. It’s the specific performance results that lie within these domains of performance that you need to always keep your eye on. They are the reason you’re measuring anything at all.
Suggestion #3: Align your performance measures and your strategic initiatives to specific performance result statements. That way you maintain the actionable link between strategic targets and strategy execution using the feedback from the relevant measures. Learn how to do this with the PuMP Performance Measure Blueprint.
Rather than just taking my word for it, look at one or two of the indexes or scores you currently use, and determine what action you might take based on what it’s telling you. Now, look at the signals in the underlying measures and see what action you would take based on what they are telling you. Is the action the same? If not, which action seems like the right action to take?
The best example I ever saw of making mistakes with indices was a productivity index developed for a Main Roads Department for wage indexing purposes. Essentially the denominator for most of the KPIs used was number of staff – the nominator being things like kilometers of road laid/repaired. The organisation was also downsizing at the time. This had the inevitable effect of making the index look really good as downsizing progressed and resulting wage increases were far higher than any other organisation in the jurisdiction. The next time they were able to do so the organisation ditched the index…probably met all the reasons above for not having an index – but really it was just a very bad design without consequences being thought through.
Ah Stacey … the simple life !!
Always agree with “keeping it simple”
I feel really lucky that you share your knowledge with me.
Your mails are very informative. It really helps me to deal with
the various PMS issues in a very pragmatic way…
Thanksa for everything ..
Take care 🙂
Connect with Stacey
Haven’t found what you’re looking for? Want more information? Fill out the form below and I’ll get in touch with you as soon as possible.
167 Eagle Street,
Brisbane Qld 4000,
Director: Stacey Barr
Speak Your Mind