IN THIS EPISODE:

Feature: 3 Things You Must Have Before KPIs Can Work.
Q&A: How do you implement KPIs efficiently?
Quick Tip: The Power of a Measures Team.

Subscribe at iTunes or listen here:

Download mp3

People don’t use the KPIs, they complain about them, and even fail to bring them to life (that is, report them). That’s because at least 3 important must-haves are missing.

MUST-HAVE #1: A climate of trust.

For a lot of people, KPI is a dirty word (well, acronym to be accurate, but let’s not get distracted). They have all manner of bad experiences anchored to that word and as a result they don’t trust you when you say that KPIs are good for them and good for the organisation.

Ask yourself this question: What will the measures be used for? And be honest when you answer it. You really need the right reason to measure before KPIs can work for you. The right reason is organisational performance improvement. The wrong reason is controlling or managing people’s performance.

MUST-HAVE #2: A clear strategy.

You can really measure just about anything and lots of people try. Or at least they’ll measure just about everything that’s easy to measure. Or that has already been measured in the past. Or they’ll measure whatever they have data for. But they’re rarely measuring what matters.

All performance measurement should be driven by the strategic direction and priorities for the organisation. Only measure what you should, can and will do something about. If you shouldn’t improve it, can’t improve it or won’t improve it, then don’t measure it.

MUST-HAVE #3: A performance measurement champion.

It’s all well and good to train people how to measure performance, to create templates for KPIs and to invest in dashboard software. But no performance measurement project ever gets real traction just by throwing resources at it. It needs to be given serious priority.

When you have a CEO or Executive who believes that measuring what matters is essential to success and is willing to shout from the mountain tops that measuring what matters is essential to success, as well as give her or his time to measuring what matters, you have a chance.

TAKE ACTION:
How much trust do you have in your organisation around performance measurement? What’s limiting it? How clear is your strategy? What would it take to make it more easily measurable? Do you have a performance measurement champion? How could you influence the right people to take on that role?

STEP 1: Know who your customers are

Gather all the relevant information you have to describe the relationship of each customer to your business or organisation, such as how much ‘business’ they give you, where they are located, what industry they are in, what their reasons are for using your services, and so on. The idea is to use this information to define market segments that group similar customers together, and help you organise your list of customers for selection into your survey sample.

For example, one of my clients was a general freight business, whose customers were everyday people who wanted to freight anything from furniture to clothes lines, from the coast to the outback.

STEP 2: Find out your customers’ needs

It’s your customers who define what ‘quality service’ means (not you!) and therefore you need to know your cusomters’ definitions of ‘quality service’ before you decide what you should be measuring in your customer survey. This means conducting a bit of qualitative research, like using focus groups, to ask your customers how they define their needs. Prioritise the top 8 to 12 of these customer-defined ‘service attributes’ as the starting point to creating your customer survey.

My freight business client’s customers cared about the care and handling of their freight, about getting quick access to their deliveries, and about the accuracy of the invoices sent to them.

STEP 3: Measure your customer service performance

Create a quantitative survey based around the 8 to 12 customer-defined service attributes from Step 2, so you can measure satisfaction levels with each one of them, along with their relative importance too. Don’t forget to include a question on the overall level of satisfaction customers have with your services. DO NOT assume you can calculate this by averaging the satisfaction with each of the service attributes.

Traditionally the freight business managers designed the questions in the customer survey: they added anything that seemed interesting to know. Surveys are impositions at the best of times, so we need to keep them short and sweet. Useful, not just interesting. So the new survey only asked questions about what customers said was relevant to them.

STEP 4: Focus on the customers’ priorities

Examining the average satisfaction ratings and average importance ratings for each of the service attributes will help you identify the priorities you need to act on. Don’t try and fix everything (aren’t you already complaining about how much you have to get through already?). Just focus on the service attributes that have the highest importance but the lowest satisfaction.

While the freight business managers believe that running on-time to their schedules was the highest priority, their customers didn’t agree. Their customers were most frustrated when they were asked to pay on invoices that were confusing and incorrect.

STEP 5: Fix your processes

Which of your business processes impacts most of the priority service attributes that need improvement? You’ll only get improvement if you fix those business processes (or at least improve them). You may even find it useful now to establish some internal measures (that is, not based on customer data) to track the processes’ impacts on those priority service attributes.

Clearly for the freight business, their invoicing process was a problem, producing inaccurate invoices as it was. They set up a new internal measure for % Invoices Unpaid Due to Errors and went about tackling the problems in the invoicing process that caused the inaccuracies.

Simple, but focused on priorities and clear action. That’s what all good customer surveys should be.

TAKE ACTION:
Do you have a customer survey now? How well does it reflect customer-defined priorities? How well is the data used to improve customer satisfaction? What can you do to make that customer survey simpler, more relevant and more actionable?

In the article, the author argued against the use of three types of customer-related performance measures, including Customer Surveys, Customer Complaints and Customer Loyalty. His points about the limitations of these measures were certainly valid, but not sufficient argument that those measures be abandoned. Here’s why:

Customer Surveys

Let’s get one thing straight up front: surveys are NOT measures!

Surveys are one tool that’s used to collect all kinds of data from customers, for use in computing a wide array of possible performance measures, including (but not limited to) Average Overall Customer Satisfaction, Average Likelihood of Remaining a Customer and Net Promoter Score.

As the author argues, many organisations send out their customer surveys and hope for a 10% response rate. He’s right in that this approach to surveying customers does produce data that is not representative or reliable enough to draw conclusions from. But that’s a problem with the survey design and implementation, and not a problem with the relevance of the measures for which the survey collects the data.

Don’t abandon customer metrics that come from survey-collected data. Just make sure your survey (including the random sample of customers) is well designed to collect the data for your chosen measures, and that it’s implemented properly by getting professional advice from a survey statistician.

Net Promoter Score

The author was also critiquing a popular measure called the Net Promoter Score as “a fairly useless metric because low scores do not provide any data to determine the reasons for customer dissatisfaction.” No single measure will provide those reasons, not even Average Overall Customer Satisfaction. The measures are for tracking trends or changes over time in a particular result. You also need to collect other quantitative and qualitative data to explore reasons why.

The purpose of the Net Promoter Score is to gauge or estimate customer loyalty by examining the intentions of customers to tell others about your organisation’s products or services. It’s not a measure of how satisfied customers are, it’s a measure of how likely they are to recommend you to their friends or colleagues. Two different results, even though they are related to one another. One measure can’t evidence all results.

Customer Complaints

Simply counting the number of customer complaints you get is not going to give you the necessary insights to improve customer satisfaction and loyalty, but it can be part of the equation. It’s got its limitations, as the author of the above article points out, like not being representative of all customers and of different complaints having different severity levels.

But the author missed the idea that measures are evidence of specific results. If you have a result to dramatically reduce customer complaints by fixing quality issues with your product or service, then tracking the number of valid customer complaints you get is a reasonable measure, even though the complaints themselves may not be representative of your whole customer base. The trend over time of the number of real complaints still carries some useful evidence.

Customer Loyalty

Just like ‘Customer Surveys’, customer loyalty isn’t a measure either. It’s a concept that actually needs a bit more definition before it becomes measurable. Is it the percentage of customers lost? Or the percentage of customers with repeat business? Or is it the length of the customer relationship? Or is it the share of the customer’s wallet that your organisation gets? It can be a number of different things, so defining it is critical in validly interpreting what it means to your organisation.

So depending on how you define customer loyalty, it may be a lag measure, or it may be a good indicator of how well you keep your customers once you attract them.

Do Universally Perfect KPIs Exist?

No. And we don’t need to have universally perfect measures. We just need measures that have sufficient strength to gauge the results we’re trying to achieve and that are feasible enough to monitor and report so it’s worth the effort and cost.

The only way to find the BEST measures or KPIs for you is to start with a very clear understanding of the performance results you’re trying to achieve, design measures that best evidence those results, and make sure you have a robust method of gathering reliable and representative data to monitor those measures through time.

Searching for universally perfect measures is a waste of time.

TAKE ACTION:
Design your own fantastic measures or KPIs – or evaluate existing potential ones – with the PuMP Measure Design How-to Kit and Template. But make sure you’re first very clear about the performance results you want to achieve. You can’t measure ill-defined goals.

What most people are really searching for is the detailed, nitty-gritty, exactly-how-do-you-do-it steps of deciding what to measure, choosing the most appropriate measures, designing new measures from scratch, implementing measures, reporting measures in a useful and usable way, and integrating measures seamlessly into decision making.

And because that’s what I was searching for back in the 1990s, in my role as Measurement Consultant at Queensland Rail, is why PuMP® was born. PuMP® is all about the performance measurement process (that’s where the ‘PMP’ comes from – the ‘u’ comes from a client who wanted to give the Performance Measurement Process a cute nickname instead of a boring acronym). PuMP® is a methodology that gives you the steps to develop performance measures. And here are those seven steps:

1. SELECT: choose what’s worth measuring

Selecting what to measure starts not with the question ‘what should we measure?’ but with first being clear about the results that matter most to you and your business. If you don’t know the performance results you’re trying to achieve, then you’ll probably too many measures that no-one finds useful, or no measures at all. And the way that most business strategy is written, it’s very hard to work out what the important results are, because of the vague language and broad terminology (for example: “We will enhance the quality, reliability, efficiency and effectiveness of our service delivery processes”).

This first step in PuMP has you doing two things specifically: we first use the PuMP Results Mapping technique to decide what results are worth measuring, and then we use the PuMP Measure Design technique to create or select the measures that are the strongest and most feasible evidence of those results. No guessing, no brainstorming.

2. COLLECT: gather data which has integrity

The process of collecting data for performance measures is critical to its integrity and can be very resource intensive. The more you can limit your data collection to what is useful, not just interesting, the better off you’ll be. So it pays (literally) to be super-specific about the data you really need for your performance measures, and not just go create a survey or form to collect a bunch of data that seems potentially useful.

There are two PuMP techniques that help maximise the benefits from your data collection efforts: the PuMP Measure Definition technique to be very precise about exactly what data each measure will need, and the PuMP Data Collection Process technique to design the steps to get the data you need without wasted time or effort.

3. STORE: manage the data so it’s quick and easy to access

Where and how you store your data directly determines what data you can access, when and how quickly you can access it, how easy or difficult it is to access and how much cross-functional use you can get out it. Most of the skill for managing performance data lies in your organiation’s IT department, but your PuMP Measure Definitions will go a very long way toward helping the IT department get you access to the data you need, with the least effort.

4. ANALYSE: turn the data into information

Analysis is the process of turning raw data into information. To make sure your performance measures are the most appropriate information you need to be almost pedantic about the analysis method you choose to answer those measures’ driving questions.

Again, the PuMP Measure Definition technique helps you make it very clear what the right analysis method is for each of your performance measures, and as such, these Measure Defintions become the blueprint or specification for exactly how each performance measure will be brought to life. No more pie charts or percentages when the real driving questions actually need a time series analysis!

5. PRESENT: effectively communicate the information

In communicating performance information, you are influencing which messages the audience focuses on. It’s vital to take care to present performance measures in ways that provide simple, relevant, trustworthy and visual answers to their priority questions. Too many people just throw performance reports or dashboards together with graphs designed to entertain rather than inform. And usually then end up misinforming!

The PuMP Reporting Measures technique helps you to design a structure, content, layout and visual design for your performance reports that syncs with decision-making and helps the real performance signals jump right off the page.

6. INTERPRET: translate the information into implication

Interpreting your performance measures means translating messages highlighted by performance information into conclusions about what’s really going on. To turn information into implication, you must discern which messages are real messages (and not all of them are!). If you’re in the habit of comparing this month to last month, or this month to a target, you’re probably drawing the wrong conclusions from your measures!

It’s the patterns, not the points, that we need to focus on with performance measures. And the PuMP Using Measures technique shows you which patterns to look for, what they mean, and how to respond to them so you don’t react to difference that aren’t real, and so you don’t miss the differences that are real.

7. APPLY: decide how implication will become action

When you have worked out what is really going on with your organisation’s performance, you are ready to make some decisions about what to improve, how much to improve it by and how to do that improving. And you want to steer clear of the typical traps people fall into when they are deciding how to respond to their performance measures.

The PuMP Using Measures technique helps you steer clear of traps like jumping to quick fixes that will fail, blaming results on things outside your control, and focusing too much on people rather than process improvement.

TAKING ACTION:
Where is your performance measurement process strong, and where is it weak? Flowchart the steps you take to select, collect, store, analyse, present, interpret and apply performance measures to find where you could get the biggest improvement in your measures for the least effort. Try this complimentary PuMP Diagnostic Discussion Tool to trigger a very insightful discussion with your colleagues.

Whether your data is quantitative (based on numbers) or qualitative (based on perceptions), it’s integrity depends on 5 widely recognised qualities.

Relevant

Make sure the data you have selected is directly appropriate to the purpose of the performance measure you selected it for. Be careful of data that seems interesting: it doesn’t mean it is relevant. Trying to gather more data than you really need, especially in surveys, can negatively impact on the other dimensions of data integrity (below).

–> Be ruthless and collect only the data you have a use for in monitoring and diagnosing performance.

Reliable

Collect enough data and collect it carefully to ensure that it is precise enough (especially if it is an estimate based on a sample) and continues to be precise enough as you collect it over time. Would you rely on one day’s rainfall to draw conclusions about annual rainfall? What about five days’ rainfall? How many days rainfall would you need to get a precise enough estimate of annual rainfall? And what would this depend on?

–> Design your sample sizes to give the reliability you need. Don’t guess.

Representative

It is important that the data you collect are observable events or characteristics that describe the full scope of what your performance measure is supposed to be measuring. This means that it is unbiased, or accurate enough. The last thing you need is for your data to tell you only what the “squeaky wheels” have to say, drowning out the valid and important and balancing views of the “well oiled wheels”. Squeaky wheels, volunteer surveys and easiest-ones-to-measure are examples of data sources unlikely to give you accurate enough data.

–> Define your population carefully, and select random samples to avoid bias.

Readable

Unless the data you collect is clearly defined, legibly presented, easy to organise for analysis, makes sense to its users and can be easily interpreted and understood by them, it won’t matter how relevant, representative or reliable it is. It just won’t be usable. The numbers need to be in a format you can use.

–> Design your data collection forms and questionnaires carefully to give you the data in the format your analysis needs.

Realistic

Trade off the degree to which your data is relevant, representative, reliable and readable with the level of resources you will need to invest to make it so. Make sure the value you get from using your data is greater than the effort you invested in getting it. Beware of the temptation to invest in sophisticated automatic data capture systems (such as bar-coding and voice recognition software) – if you haven’t got a simple manual system working well first, then these systems are likely to cost you much, much more than the savings they appear to promise.

–> Pilot test your data collection processes to be sure they will deliver cost-effective data.

TAKING ACTION:
If you have a performance measure or KPI that triggers more debate about data quality than it does about performance levels, then use the 5 Rs of data integrity to work out where the data collection process can be improved.

The third thing I don’t like about it is this:

CHALLENGE 3: The Balanced Scorecard is not a performance measurement methodology.

How dare I utter such a blasphemous suggestion!

But I truly believe it. The Balanced Scorecard is, in my book, far more a strategy design methodology than a performance measurement methodology. And here’s why: A performance measurement methodology has to go much further than just suggesting how to determine a balanced and cause-effect linked strategy.

A performance measurement methodology has to help you design and implement and use performance measures, too:

1. It has to help you find meaningful measures, particularly when the strategies seem at first to be immeasurable. There are many Balanced Scorecards that are filled with lame, vague measures when they don’t have to be.
2. It has to help you nut out the details of your measures, so they can be implemented as intended. Too many of our performance measures are poor substitutes for what we originally intended them to be, because not enough thought went into the appropriate calculation and data requirements.
3. It has to help you analyse and report your measures so they clearly and engagingly tell the story of actual performance.
4. It has to help you engage people to measure performance willingly and honestly, and as easily as possible so the measures have the best chance of truthfully telling the story of performance.
5. It has to help you validly interpret the quantitative information that the performance measures are providing, so decisions are based on patterns and trends instead of knee-jerk reactions to individual points of data.

The Balanced Scorecard does nothing to help you with these challenges. It isn’t a performance measurement methodology – it’s a strategy design methodology.

But before you think I’m on a one-woman mission to bag the bejesus out of the Balanced Scorecard, let me say this: I don’t advocate you don’t use it, I just want you to be aware of its limitations despite its popularity, and make sure you take from its strengths and compensate for its weaknesses.

TAKING ACTION:
Do you have a step-by-step performance measurement process to populate your Balanced Scorecard with meaningful measures, and then implement and use those measures to execute and achieve the strategy implied by your Balanced Scorecard?

The second thing I don’t like about it is this:

CHALLENGE 2: The Balanced Scorecard perspectives are too limiting.

The four perspectives that comprise the Balanced Scorecard are Financial, Customer, Internal Business Processes, and Learning and Growth. And these four perspectives work in a cause-effect flow, from Learning and Growth up through to the Financial perspective.

The idea is that you design your strategy across these perspectives, and you choose measures (KPIs) aligned to this strategy, and hence you have your Balanced Scorecard.

Do Four Perspectives Developed Over 15 Years Ago Still Apply?

In an age where social responsibility, environmental responsibility and systems thinking are driving much of our thinking about what matters in managing organisational success, I struggle to accept that all that matters in a strategy can fit into the Balanced Scorecard’s four perspectives.

And indeed, in the numerous situations where I’ve seen the Balanced Scorecard used, that’s exactly the way people behave: they try and fit their strategy into it. Alternatively, they create their own perspectives, often around Critical Success Factors that emerged from their business scanning and SWOT analysis.

Alternative Ways To Design A Corporate Strategy.

One model I really like is designing perspectives around key stakeholders (like customers, shareholders/owners, strategic partners, employees, and the community) and their definitions of the value the organisation or company provides them. It’s a great model if social responsibility is important to, at least as much as profit is.

The Performance Prism is one such stakeholder model.

Another model that’s quite common is TBL or Triple Bottom Line, which moves away from profit as sole definition of organisational or company success and brings in a new idea of balance with the People and Planet bottom lines companioning the Profit bottom line.

Perhaps if you apply some systems thinking, examine your internal and external business environments, and take your SWOT analysis seriously, you will see what really matters for your organisation’s or company’s success. And if you then explore how each business process and function impacts those strategic results, you’ll more naturally cascade the strategy.

One More Challenge…

In part three of this series, I’ll discuss the final thing I don’t like about the Balanced Scorecard, and again will suggest some tips for compensating for this challenge.

TAKING ACTION:

In using the Balanced Scorecard, what important results are you ignoring because they don’t fit? What results are you focusing on and measuring, because you think you should have something in each of the four perspectives of the Balanced Scorecard? Let’s continue the discussion, at the Measure Up blog!

Certainly, on account of the Balanced Scorecard, we’re now seeing the measurement of non-financial results rather than just the financial, and we’re seeing strategies laid out in logical and cause-effect linked plans designed for execution rather than shelving.

But a few challenges continue to baffle those that embrace the Balanced Scorecard way. One of the challenges is easy and quick to remedy within the current Balanced Scorecard theory. But the other two, I believe, require a more radical re-think.

In this first part of a three part series, we’ll look at one of those challenges that does indeed need a more radical re-think.

CHALLENGE 1: The Balanced Scorecard is hard to cascade meaningfully.

You might argue with me on this point, because part of the Balanced Scorecard’s claim to fame is it’s focus on strategy execution and cascading strategy to operational levels. But those famous four perspectives that were the revelation of this framework are also the limitation on meaningfully cascading strategy.

What Happens Is “Mini-me” Syndrome.

I call it the “Mini-me” syndrome (inspired by the Austin Powers movies), where what ends up being cascaded are localised scaled-down copies of the corporate scorecard. Each department or team has the same perspectives as the corporate scorecard, almost the same strategy map, but tailored to the scope of their work.

If injury reduction is in the corporate scorecard, then every department and team has injury reduction in their scorecard: even those departments where injury risk is infinitesimal. If cost reduction is in the corporate scorecard, then every department or team has cost reduction in their scorecard: even those departments (like Human Resources or Process Improvement, whose costs must increase in order for other areas’ costs to decrease.

That’s not true cause-effect thinking, and it leaves many managers and employees bemused and cynical about having to measure things that don’t really matter to them, and that don’t really focus on their specific and unique contribution to the corporate direction.

Additive Thinking Is Not Cause-Effect Thinking.

When the focus is on maintaining the four perspectives in everyone’s scorecard to link up to the corporate scorecard, the attention has moved away from where it needs to be: focusing on the performance results and process improvements that have the highest leverage to achieve the corporate strategy.

What happens instead is a collection of additive scorecards, where you can add up or combine the metrics from scorecards across the departmental tier, and end up with the values for the corporate scorecard. Likewise, you could add up the add up or combine the metrics from scorecards across teams within a department, and end up with the values for the departmental scorecard. This isn’t cause-effect thinking. It’s additive thinking.

Cascade True Cause-Effect, Not The Scorecard.

To apply true cause-effect thinking, we have to let go of structure. We have to openly explore and analyse how the performance of a part truly does impact on the performance of the whole. The four perspectives of the Balanced Scorecard don’t encourage that open exploration and analysis, and that’s why we have the Mini-me problem.

I haven’t found a sensible and easy way to help departments and teams cascade the Balanced Scorecard in a way that’s sensible for them and truly aligned to the corporate direction. Instead, we use a more open approach called Results Mapping, which encourages them to start with a conversation about the corporate direction (or scorecard) and explore the question “How and where do our results and our processes most impact on the corporate direction?”

Two More Challenges…

In parts two and three of this series, I’ll discuss two more things I don’t like about the Balanced Scorecard, and suggest some tips for compensating for these challenges also.

TAKING ACTION:

Where are you trying to cascade the Balanced Scorecard? Is it making sense to the teams it is cascading to? Is there anything in their scorecard that isn’t really that important, or anything missing that actually is important? What questions are you asking to guide the way that strategy is cascaded in your organisation or company?

How do you translate results so intangible as employee morale or service quality or corporate image into solid, robust measures?

The framework described here is an excerpt of the How-to Kit: How to Design Meaningful Performance Measures, which provides a systematic approach for taking almost all of the pain out of the challenge of finding the right measures.

STEP 1: Begin with the end in mind.

Performance measures are objective comparisons that provide evidence of an important performance outcome. It is of the utmost importance to decide which outcomes are most worth tracking right now. As the first step in deciding how to measure an outcome, write down what the outcome is, what the difference is you are trying to create (and thus want to track using a measure). Focus on one outcome at a time.

STEP 2: Be sensory specific.

When you have the end in mind, you are ready to get a handle on what specifically about your outcome you will measure. This is where you take care in your choice of words to describe the outcome as concretely as possible. Use “sensory” language – the language that describes what you and others would see, hear, feel, do, taste or smell if your outcome was happening now. Avoid those inert words that we so often see in our goal and objective statements, such as: efficient, effective, reliable, sustainable and quality.

STEP 3: Check the bigger picture.

Check the bigger picture for what could happen if you measure your outcome. What level of control do you have over achieving it? What might the unintended consequences of measuring the outcome be (both the positive and the negative)? What behaviour would the measures drive? Which other areas of performance might be sabotaged or limited? This is your first chance to change your mind about what’s most worth measuring.

STEP 4: What’s the evidence?

Now, get ultra specific and figure out what the potential measures are that could let you (and everyone else) know that the outcome is being achieved. For each of your sensory rich statements from step 2, what could you count to tell you the extent to which it is occurring? Which of these potential measures would be the optimal balance between objectivity and feasibility?

STEP 5: Name the measure.

Naming your performance measures marks the point at which you know exactly what you will be measuring. Be succinct and informative and deliberate, as you need to be able to continually and easily identify each measure as it moves through the steps of being brought to life and being used in decision making.

TAKING ACTION:
Create your own measure design template based on these 5 steps (or save time and use mine, which includes examples and more detailed instructions, in the How-to Kit: How to Design Meaningful Performance Measures). Now use your measure design template to start designing measures for the tricky goals and objectives and results and outcomes you’ve struggled to measure thus far. Practice makes perfect!