Question 1: What is performance really doing?

This doesn’t mean what is performance doing today, or what did it do last week? It means what is performance mostly likely to keep on doing if it’s left alone? To answer this question, your performance measures need to be displayed in graphs that show the natural variability in performance over time. That’s a new concept to many people, but it’s so very simple to do it.

In your performance reports, XmR charts are the best way to display performance measures. These charts validly and visually flag the signals of true change in performance: a sudden shift, a gradual trend, a change in variation. To learn about XmR charts, get Donald Wheeler’s fabulous book “Understanding Variation: the key to managing chaos”. It’s written for management, it’s easy to understand and you can use this XmR chart for any type of performance measure that you track regularly through time.

Question 2: Why is it doing that?

When you see a signal in your performance measure, like a sudden shift in the wrong direction, it’s tempting to jump straight to solutions. Train staff… hire consultants… invest in technology… educate customers… write policies. But usually these “fixes” fail – you won’t see the performance measure respond very well, if at all.

Fixes fail when they aren’t designed to target the true causes that underly the signals we see in our performance measures. So it should go without saying that before you even think about a solution, you ought to have interrogated your measures and other related data to find out the cause behind the signal. This means analysing the work process that is being measured to identify potential causes, using data to verify which causes are correlated with the signal in your measure, and including a summary of this cause analysis in your performance report.

Question 3: How should we respond?

When you know the cause of the signal in your performance measure, you’re much more likely to look for the right solutions. But there’s more than one way to solve most problems, so performance reports that are most useful are ones that include some options for solutions, along with the pros and cons for each, so the report’s audience can make an informed decision about the best way to respond to the performance measure.

TAKE ACTION: It might be time to take stock of your performance reports and see how well they answer these 3 questions. While you’re at it, find out from users how useful and how useable they feel the reports are. Then design a fresh performance report template based around the 3 questions. For a short cut in the form of instructions and a ready-made report template, take a look at the PuMP How-to Kit for Designing Useful and Usable Performance Reports at www.staceybarr.com/products/reportdesign.html.

IN THIS EPISODE:

Upcoming Events: Interested in a 2012 Performance Measure Blueprint Workshop? Let us know…
Practical Tip: Three questions to design your KPI reports
How-to Resources: How to design useful and usable performance reports…
Your Questions: Chris asks how to present KPIs with minimal effort.
Getting More: Five Steps For a Fast Performance Dashboard, Making Sure Your KPI Graphs Don’t Suck, WOW! What a Cool Performance Report!

Subscribe at iTunes or listen here:

Download mp3

To get the dashboard project moving again, five simple steps are all you need:

STEP 1: Focus only on your top 2 to 3 priority performance results.

Stop trying to do it all at once, perfectly. It’s expensive, it’s time-consuming, it’s momentum-dampening. You’ll get to success sooner by taking the quick and easy route to set up simple, bare-bones dashboards that do the job with inexpensive applications like KPI Dashboard, Tableau and even Microsoft Excel. Once you’ve finished these 5 steps for these first priorities, focus on the next 2 to 3 most important performance results, and keep building and improving in bursts.

For my own business, I use a Results Map to capture all the performance results that make up my strategy, and it helps me to work out the most important ones, the results that are pivotal to success and need to be measured first and foremost.

STEP 2: Choose a few performance measures for those priority results.

There’s no prize for using all the KPIs you can think of to track a particular performance result. In fact, there’s a penalty and it’s your attention being spread too thin to make any kind of performance improvement at all, or your attention being pulled away from your unique priorities. Think narrow and deep. Go for the truly relevant and insightful performance measures.

If I ever get stuck and can’t figure out what the best measures are for a performance result, the five-step Measure Design template works a treat. It makes you think about what your result really looks like in tangible terms, and that’s the key to finding the right measures.

STEP 3: Define exactly how those priority measures are calculated, and from what data.

Lame measure names won’t cut it. “Customer Loyalty” isn’t a performance measure. “The percentage of customers who have purchased from us more than 3 times in the past 6 months” is a measure. One of the biggest time-wasters in setting up a performance dashboard is trying to figure out exactly how the measures should be measured. That sounds tautological, and it is. If you can’t figure out how to measure something with real data, then it ain’t a measure yet!

Before I build my dashboards, I define each of my performance measures and keep these definitions in a Performance Measure Dictionary.

STEP 4: List the data sources for priority measures, and set up links in your dashboard to that data only.

A lovely man named Siva once had dozens (some say hundreds) of special spreadsheets where very key performance measure data was kept for a freight business. Only he knew how the spreadsheets worked, and where they were saved on the server. And I think he has sinced retired (or won’t be far off). Doesn’t it make you shudder?

My performance dashboard is built with Tableau software, inexpensive and easy, but most importantly for me, able to link directly to my core business database. When my assistant updates my business data each month, my dashboard updates automatically. Love it!!

STEP 5: Systematically create the graphs in the dashboard for each priority measure, and arrange them on a single dashboard.

Forget silly dials and gauges. Forget pie charts and 3-D bar charts. (Even though it’s near impossible to find an example dashboard without these useless gizmos!) The most effective way to monitor your performance measures is with a line chart that tracks changes over time. That’s what you’re managing when you manage performance: moving performance closer and closer to target as time goes by.

I arrange my performance measure line charts onto my dashboard in segments that correspond to my priority performance results. After all, you’re using the measures to know if you’re acheiving the results. That’s one of the 5 principles of designing useful and usable performance reports.

TAKE ACTION:
Whether you choose Tableau, KPI Dashboard, your existing dashboard/scorecard application, or even just Microsoft Excel, use these five steps and bring your most important KPIs to life *this week*. As Dr Poh, one of my readers, recently emailed me, “If not I, then who? If not now, then when?”

IN THIS EPISODE:

Feature: 5 Steps For a Fast Performance Dashboard.
Q&A: Should dashboards be used only for KPIs that can be reported on daily or weekly?
Quick Tip: A review of KPI Dashboard.

Subscribe at iTunes or listen here:

Download mp3

IN THIS EPISODE:

Article: 3 Things You Need On Every KPI Graph.
Q&A: How to get Executives understanding different methods of reporting performance results.
Quick Tip: How to choose the right graph type for your KPI or performance information.

Subscribe at iTunes or listen here:

Download mp3

FEATURE 1: Your performance measure in time series

This month compared to last month, this month compared to the same month last year, this month compared to target – all are limited and risky, misleading comparisons to make. Why? Because you have no context within which to assess if the difference you’re looking at is typical or not. There will always be differences, but the only way to see which ones are worthy of your response (or explanation) is to see the data in the full context of a time series.

Plot your performance measures in time series, with as much history as you can reasonably get your hands on. Around 20 values is good, such as the last two years’ worth of monthly values.

FEATURE 2: The measure’s mean line

The signals about performance lie in the patterns, not the points of data. Two points is not a big enough sample to determine if there’s a change. Adding a mean line to your time series, calculated from the first 10 points or so and then kept constant, will give you a visual benchmark to determine when future points of your measure are behaving differently from the past.

You’re starting to build what’s called a control chart, and there are statistical rules that help you know when a signal is there and when to recalculate a new mean line. For example, one of the signals of change is a run of 7 points of your performance measure on one side of the mean line. You can then calculate a new mean line using those 7 points.

FEATURE 3: The measure’s limits of natural of variation

Everything varies. Everything. And performance measure values tend to vary quite a bit, from month to month or week to week or quarter to quarter. Just because one point varies more than 10% from your average, doesn’t mean you have a problem. It depends on how much natural variation is in your measure to start with.

Control charts include a measure of the natural variation. For example, if a point falls outside the limits of natural variation on the chart, then you know something has happened out of the ordinary and it’s worth following up. Limits of natural variation also help you pick up signals from only 3 or 4 points of data in a row (you don’t always have to wait for 7 points to behave differently).

BUT: You do need to learn how to use these control charts.

It doesn’t come naturally, and people who don’t take the time to learn make mistakes in constructing and interpreting them. But they’re really easy to learn and VERY worthwhile learning.

The most convincing and clearly articulated resource that I’ve ever seen about using charts like these for management reporting is Donald Wheeler’s book, Understanding Variation: The Key to Managing Chaos. If you consider yourself interested in performance measurement and performance improvement, this book MUST be in your personal toolkit.

TAKE ACTION:
Ratchet up your knowledge on control charts for management reporting. Get a copy of “Understanding Variation: The Key to Managing Chaos” and then go find some performance data, and have a go at using a control chart to reveal the true signals.

PRACTICE #1: Use two or three related measures, not just one measure to understand a specific result. Staff Turnover is not a sufficient indicator of staff engagement, but combine it with measures like the percentage of staff intending to leave within the next 6 months and staff satisfaction with their worklife, you get a less biased picture.

PRACTICE #2: Don’t tick & flick – consider all your measures as important characters in the same performance story. Don’t you think that optimising on-time running and safety performance is much wiser than trying to maximise each of these, independent of the other?

PRACTICE #3: Consider qualitative indicators as well as quantitative measures to add more life to the story of performance. Just like a series of video case studies or photos of smashed up cars give sensory context to improvements over time indicated by road crash severity metrics.

PRACTICE #4: Look at your data in more ways than one - the average is only one small part of the plot. A fuller understanding of customer satisfaction comes from examining the variation in satisfaction ratings, the change in satisfaction levels over time, the correlation between satisfaction levels and revenue – not just the average satisfaction rating.

PRACTICE #5: Report causal analysis and hypotheses along with the performance results, to give the story somewhere to go. If cycle time is blowing out, then find out the events that likely triggered the shifts in cycle time, or find out on which activities most of the cycle time is being spent.

PRACTICE #6: Ask your data questions to draw out more of the story. Why aren’t costs staying within budget? Which parts of the budget are blowing out most? Where is budget being managed well? What if we cut back spending on this?

When you think about putting your performance measures to use to improve performance, think systemically: do what you can to draw out the story in the performance data. Stories engage people, and when it comes to numbers and statistics and measures, we usually have to do all we can to get people engaged!

TAKING ACTION:
Think of one performance measure for which you have plenty of historical data. Collate that data, and start looking at it using different tools, to uncover its story. Try a time series graph to see if it’s changed over time, a histogram to see its variability, a scatter plot to see how strongly it is associated with other measures.

We all know that one of the big reasons people will fight tooth and nail to avoid measuring performance is that it’s hard and boring (at least from a newbie’s or cynic’s perspective).

And it’s no help that people see performance measurement taking so damn long to get implemented.

So it’s probably no surprise that my PuMP Performance Measure Facilitators say the best part of their program is when they produce the first properly designed performance report with their teams.

One PuMP Facilitator said it was the point that it all came clear to his team just why they were putting the effort into measuring.

As a performance measure practitioner yourself, one of your own measures ought to be the cycle time to get from “We need some measures,” to “Wow, what a cool performance report!” And track that cycle time for each performance measurement implementation assignment you undertake within your organisation.

So how can you reduce this cycle time from the typical year or more, to mere weeks? Try these ideas on for size:

1. focus your team on measuring just one goal or one performance result first time through
2. treat it as a pilot test – don’t try to perfect and complete it all, just get a first cut measure established
3. give a deadline to each step in the measurement process, and get as good as you can within the deadline rather than waiting for perfection before starting the next step
4. make sure your measures team has allocated enough time to work on measures each week
5. have the measures team’s manager regularly pop in for updates

The sooner you have your colleagues using measures they value, the faster you’ll ramp up the performance culture and the faster everything else will improve!

YOUR CHALLENGE:
Reflect on how long, on average, it’s been taking your colleagues to get from wanting measures to actually having those measures ready to use, in performance reports. Then set a target to reduce that time by 50% and set about streamlining your measure creation process!

And do you wonder why people aren’t really getting much value from measuring performance?

Because performance measures are feedback, you need to track them often enough for the feedback to be of any use in guiding your decisions and actions to improve performance.

So here’s a checklist to help you decide whether you should measure a result on a weekly, monthly, quarterly, or annual basis:

__ How SENSITIVE is the performance result to change? If you were to go make an improvement now, how long would it take before you saw the effect? For example, if it takes years to see change happen, like a staff safety culture or community attitudes to water use, you can probably get away with measuring six monthly or annually.

__ How URGENTLY do you want to know if the performance result has shifted? Is this a result that needs to be improved in the next six to 12 months? If so, it probably needs to be measured at least monthly!

__ How COSTLY would it be to measure the performance result more frequently? Is this worth the benefit of having more frequent feedback?
Could you improve performance so much that the cost of measuring more frequently is far outweighed by savings or added value?

__ How ACCURATELY do you need to measure the performance result? By taking smaller samples more frequently, you can reduce the cost of measuring often. But with smaller samples comes less reliability, so take care. If you just need a good indicator of the trend or change over time, smaller samples can work a treat.

YOUR CHALLENGE:
Think about the kind of response or action you designed your measure to inform. Are you measuring it frequently enough to inform you before you act, or only frequently enough to tell you after you act?