IN THIS EPISODE:

A Case Study interview with Jerry Stigall, a Strategy Professional who led Douglas County to win the Balanced Scorecard Award for Excellence.

Subscribe at iTunes or listen here:

Download mp3

Using rail as their primary mode, the freight business featured in this case study hauled bulk freight, such as grain or livestock, for their customers.

Their performance measurement challenge

Via their customer survey, the freight business identified that one of the top three priorities for improvement was the accuracy of their billing process. Customers were not paying bills because they were based on inaccurate or confusing rates.

The challenge was to measure and improve three specific performance results:

• timely payment of invoices
• revenue collected for all consignments handled
• ease of understanding of invoices, statements and adjustment notes

How they overcame these challenges

A small team from the freight business worked together, using PuMP, to:

• Define their billing process and its purpose.
• Analyse their billing process to identify where the causes of problems were.
• Develop performance measures to monitor the improvement of the billing process.

Here’s what they did…

Process Result Measures:

Measures of the above three performance results were called ‘Process Result Measures’. The team decided on six:

• Value of Overdue Accounts = The total amount of revenue currently outstanding for accounts that have not been paid by their due date.
• Lateness of Overdue Accounts = The average number of days that overdue accounts are overdue by.
• % Freight Movements Billed = The total number of freight movements (consignments) that have been billed as a percentage of the total number of freight movements that have been consigned.
• Customer Satisfaction with Invoice Accuracy = The average satisfaction rating provided by customers about the accuracy of invoices they receive from us.
• Bad Debts as a % of Revenue = The total amount of revenue for invoices classified as bad debts, as a percentage of the total amount of revenue received.
• Cost of Billing = The total labour costs associated with preparing, sending and following up invoices.

The Process Flowchart and problem areas:

To understand how the billing process might be improved, to improve performance as defined by their six Process Result Measures, the team created a process map:

Process Problem areas:

After mapping the billing process, the team systematically stepped through it to identify the problems that currently prevented good performance in the Process Result Measures:

• Freight consignment notes are not always supplied when they should be, and freight is shipped without them.
• Freight consignment notes go missing when they are paper-based.
• The wrong charges can be applied when the data about exactly what freight and what amount of freight is not accurately provided by the customer.
• Accounts should never get to this stage, but there is a considerable number that are in dispute over what has been invoiced and the charges applied.

In-Process Measures:

Before they decided how to fix these billing process problems, the team designed a few measures to track these problems, so they could tell whether or not their fixes were working. They called these measures ‘In-Process Measures’:

• % Freight Consignment Notes Not Supplied = The number of consignments where a consignment note was not supplied as a percentage of the total number of consignments.
• % Freight Consignment Notes Electronic = The number of consignment notes that were supplied electronically as a percentage of the total number of consignment notes supplied.
• Accuracy of Freight Data = Number of errors found in quantity or type of freight that would affect how the freight was charged, divided by the number of freight consignments.
• Value of Accounts in Dispute = The total amount of revenue associated with accounts that are currently being disputed by customers.

With this collection of six result measures and four in-process measures, the team could focus very directly on fixing the problems that mattered most, and making sure that their efforts were improving the in-process measures, and in turn improving the process result measures.

TAKE ACTION: Where can you use process analysis – including identification of process measures – to help you improve a strategic outcome or goal?

And maybe like me, you also think it’s hogwash.

Goals are promises to close the gaps between as-is performance and should-be performance that are too wide to tolerate any longer. Strategic goals are the gaps that are the most important to close, such that the organisation takes the largest possible steps toward excelling at what it exists to do.

Good goals are those whose gaps are observable in some way. If you don’t know how you’d recognise the gap shrinking, you don’t have a good goal yet. And that’s the problem with the typical lofty goals you see in strategic plans. No-one understands, or shares the same understanding of, what the gaps look like.

What is the current gap between as-is health of citizens and should-be health? It makes no sense to ask that question, because ‘health’ is too broad. It isn’t just one gap; it encompasses many observable gaps. Disease, injury, fitness, nutrition, depression, substance abuse, stress, and more.

So the question is not how to find KPIs for lofty goals. The question is how to turn lofty goals into measurably meaningful goals. The key to doing that is to explore the possible observable gaps, and then choose the one or two that are absolutely the most important to close. If you don’t do that, and keep those goals lofty, nothing will change in a measurably meaningful way.

TAKE ACTION: Check if you have any lofty, hard-to-measure strategic goals. If you do, take a step back and ask what the goal really means. Try to describe the evidence of it being attained, of the gap between as-is and should-be closing. If you have lots of different types of evidence, then the goal is probably too broad. So break it down into specific and evidence-able gaps, and then work out which gaps are the most important to close. Then just measure those gaps.

My eyes were opened to consequences beyond animal suffering when I read about the mass-production farming of chickens, in “The Ethics of What We Eat” by Peter Singer and Jim Mason. To maximise profits, these companies maximise the number of birds per square foot of space (in many operations, chickens get about the size of an A4 sheet of paper to live their lives in). Aside from the birds’ suffering, there are some very significant costs to local communities and ecologies.

The ammonia from the chickens’ waste products makes going outside unbearable when the wind blows in a specific direction. The ammonia causes irritation and health problems to people. When the waste products are flushed from the factory floor, they run off into nearby streams and waterways, and kill the aquatic life. These are the costs that don’t make it into the companies’ financial statements, costs that communities and the environment pay (now and in the future) to subsidise those profits.

There are still many companies that don’t take responsibility for the lifecycle of the products they make. Electronic appliances like toasters and video players and outdated laptop computers often aren’t able to be recyclabled. And no-one wants to reuse them when they can buy the latest and greatest for next to nothing. So they become land fill.

Even that habit of calling employees ‘our greatest asset’ smacks of a company that treats people like equipment or property: maximise the return on investment. Get as much productivity as possible for the least amount of financial reward. Child and slave labour in industries like coffee and diamonds are of course the extremes of this view, but it’s still an attitude that many familiar companies and organisations have of the people who contribute their time, skill and effort.

If human endeavours like business don’t exist to make the world a better place, can we really judge them as successful? How can people enjoy the rewards of high profits when others have had to suffer to make those profits possible?

We really need to move away from a singular focus and single measures of success. Surely a truly balanced scorecard is one where the perspectives of every stakeholder affected by a company define success, and guide what that company measures, monitors and manages to pursue that success?

TAKE ACTION: Is your strategy based on just one or two stakeholders’ values? Who or what else is affected by your business? As a starting point, think about shareholders, customers, employees, suppliers or partners, local communities and the environment.

Bec Bach is a Performance Analysis and Reporting Consultant with the Department of Education and Children’s Services in South Australia (DECS).

Bec used the PuMP technique of Results Mapping with one of DECS’ country schools. This school had fairly high levels of disadvantage, and fortunately it also had the most enthusiastic and positive staff that Bec has worked with. This school underwent a review that recommended specific improvements to their performance.

As a result, the school wanted to update their 3 year improvement plan and invited Bec to help with their targets.

Taking a necessary step sideways to make sense of what to measure

Bec recounts what happened when she introduced PuMP to the whole Leadership Team at the school:

“The first time that I took the Leadership Team through the notion of PuMP I could see that there were a few people who were struggling with it, which was good because they were really honest and open with me. But I really liked the way that the Assistant Principal summed it up. He said ‘Look, you know we asked you to come to help us with something. You’ve come and presented us with something else, but you know what? We’ve taken a step sideways and we’ve actually got a broader view.‘”

Should you really measure 115 priorities?

The enthusiasm, in fact, turned out to be a challenge for Bec. They wanted to improve everything! Looking at their improvement plan, they had 5 priorities. But within each of those priorities there were about 4 or 5 goals. And they also had a plan for the Junior School, the Middle School, the Senior School and also for the Aboriginal Education Team within the school.

So when Bec started them on Results Mapping, bringing it all together into the one visual map, they found for one priority (about literacy) they had 115 goals to achieve in 3 years. And they were wondering why it was a struggle to get there!

Their plan had never been represented visually before, and this was the biggest impact of the Results Map for this Leadership Team. All their goals suddenly became connected together, and showed them how much improvement work they were trying to do. With 115 goals, it’s just impossible to go really deep and thoroughly into that improvement work.

Focusing measures on things that need improvement versus maintenance

Bec said to the Leadership team:

“Yes they are all important, but what are the things that are most important, what are the commonalities between the Junior School, the Middle School and the Senior School? How can you collectively bring some of those goals together and make them a school goal?”

Bec explained that one of the things that the school found a bit hard was to take things out of their plan. They felt if they take it out of the plan, then they’re not doing it. She suggested to them:

“Well, hang on you know, you can’t improve everything, so work out what you want to improve. It doesn’t mean that you’re not doing things, because things can go into a maintenance mode.”

It turned out to be a really helpful process. They ended up with about 4 goals for the school as a whole, and then a number of different measures. For example, with their improved literacy outcomes, they would use a different measurement in the Junior Primary than they would in the Middle School or the Senior School. Then they could link the work in the classroom with the school goals. Bec said:

“And that’s again the beauty of the Results Mapping, because for the first time people could really see the impact of what they did in their classroom and how that influenced a school goal, which helped to then achieve a priority. That visual connection was so strong and so important for people.”

TAKE ACTION: The best way to get out of a rut is to stop, and take a sideways step. Getting out of performance measurement ruts is no different. What’s the deepest performance measurement rut that your organisation faces? More immeasurable goals? Worsening disengagement from people? Continually selecting lame measures? Stop, and take a sideways step out of the rut by deliberately adopting a fresh approach.

IN THIS EPISODE:

A Case Study interview with Bec Bach, a Performance Analysis and Reporting Consultant who helps South Australian schools use data and evidence for improvement planning.

Subscribe at iTunes or listen here:

Download mp3

An index is the term we give to a measure that is computed by taking a weighted sum or weighted average of a collection of other measures.

For example, you might take a dozen or so different measures to do with productivity, and create a weighted average of those measures to give you an Overall Productivity Score. Or you might create a Business Unit Performance Index for each of the business units in your company or organisation.

It might sound like a good idea, but it isn’t. Here are three reasons why indexes and scores are a bad idea:

Reason #1 is that you lose focus. If you’re using an index because you have too many measures to digest in one sitting, then the problem is that you have too many measures! Performance measurement is not about measuring everything you can, it’s about measuring only those things you can and should and will do something about.

Suggestion #1: Test the value of each of your performance measures using this checklist of criteria of excellent performance measures.

Reason #2 is that you lose the real signals. The signal your index gives you will be a diluted average of the signals of all the underlying measures. If you have one underlying measure tracking very well and another underlying measure tracking very poorly, they will average out in the index.

Suggestion #2: Rather than summarising every measure into an index or score, use simple formating to highlight the signals in each underlying measure, to make it easy to focus just on those measures that need attention. Learn how from dashboard guru, Stephen Few, at www.perceptualedge.com.

Reason #3 is that you lose sight of the tangible results you are trying to monitor. When you look at an index or score, it’s usually a rolled-up summary of a broad performance area, like productivity or customer service or financial performance. But these broad performance areas aren’t tangible or actionable. It’s the specific performance results that lie within these domains of performance that you need to always keep your eye on. They are the reason you’re measuring anything at all.

Suggestion #3: Align your performance measures and your strategic initiatives to specific performance result statements. That way you maintain the actionable link between strategic targets and strategy execution using the feedback from the relevant measures. Learn how to do this with the PuMP Performance Measure Blueprint.

TAKE ACTION: Rather than just taking my word for it, look at one or two of the indexes or scores you currently use, and determine what action you might take based on what it’s telling you. Now, look at the signals in the underlying measures and see what action you would take based on what they are telling you. Is the action the same? If not, which action seems like the right action to take?

This question is a wise one, because many people still don’t realise that it is essential for planning to come before measuring. We don’t measure for measurement’s sake. We measure to get objective feedback about whether the results we want are moving in the right direction.

Most planning processes start with some kind of business scanning. A SWOT analysis is a popular and simple way to do this. It guides you to explore your internal strengths and weaknesses, and to explore the external opportunities and threats that all impact on fulfilling your mission and pursuing your vision.

When I the Balanced Scorecard Institute’s Professional Certification course, it taught me the importance of actually analysing the SWOT, not just compiling the SWOT. By looking at the collection of strengths, weaknesses, opportunities and threats as a whole, you can sift out three or four strong themes that become your strategic focus.

Some will call these themes Critical Success Factors (CSFs), Key Result Areas (KRAs) or Strategic Priorities, but in Balanced Scorecard parlance, they’re called Strategic Themes. I don’t reckon the label matters as much as their ability to focus you on what truly drives your mission and vision right now.

Within each of your Strategic Themes or CSFs, you can then define several specific performance results that make those themes tangible. When you articulate your performance results in language an 8-year old would understand, they will be measurable. This means it will be far easier to select one or two performance measures for each performance result, measures that will be objective evidence to monitor your progress in achieving each of your results.

When you combine a performance result with its performance measure and a target and a timeframe, what you have is a true goal. Only after you have these true goals does it make any sense to start talking about the initiatives or projects that have the greatest leverage to achieve those goals.

TAKE ACTION:
Involve the team in a SWOT analysis. Allow 1 or 2 hours to collate strengths, weaknesses, opportunities and threats. Then allow another hour to identify 3 or 4 Strategic Themes (or Critical Success Factors). The PuMP Performance Measure Blueprint can then give you the steps to define the performance results and the best performance measures for them.

Oh, and a word of warning: No list of criteria is ever complete or ever totally correct. So as you read these and apply them, tune into the purpose of the criteria if the prescription just doesn’t fit.

1. Does it have clear link to strategy?

For your measure to be excellent, it must have a clear line of sight to your business direction – to your strategic goals or priorities. If you are measuring something and improving it won’t make any significant contribution to achieving your strategy or goals, do you really need to measure it? (Idea: design your measures at the same time you formulate your goals.)

2. Is it owned by someone?

Unless your measure is officially owned by someone – not a department, not a team, but a person – it’s super likely that it isn’t being used to make performance improve. It may not even be properly reported. Measures need an owner to make sure it is reported and used for the benefit of the business. (Idea: find a person who has enough authority to respond to the measure.)

3. Can it be brought to life?

Vague ideas, surveys and kooky acronyms are not measures. A measure needs to be spelled out in enough detail that you can know exactly how to calculate it, how often and from which data. (Idea: define the details of bringing your measure to life to test its viability.)

4. Are you able to track it regularly over time?

Before and after measures (which is what annual measures usually end up being) don’t give enough feedback to manage improvement efforts. Your measure must be tracked regularly enough (such as monthly or weekly) to give you clues about whether your improvement efforts are working, before it’s too late. (Idea: measure more frequently or find a lead indicator that you can measure more frequently.)

5. Does it give you more value than it costs?

Measurement doesn’t happen for free. But you need to be confident that the costs associated with its design, data collection and reporting are less than the benefits it brings to your business, which is usually through decision making and the resulting improvements you get for your bottom line or your stakeholder value. (Idea: remove waste and duplication from data collection and reporting, e.g. use sampling instead of measuring it all.)

6. Do users understand it?

If your measure is a convoluted index of other measures, or it’s calculation is difficult to explain in everyday language, it can make using it too daunting a job. (Idea: borrow from Ockam’s Razor and find the simplest measure that can convey the needed information.)

7. Does it inspire the right behaviour?

If your measure is not encouraging people to choose performance-improving behaviour over sweep-it-under-the-rug behaviour, it’s a candidate for throwing out. The measures that are the most potent in improving business performance are those that make it obvious – even unconscious – the right actions to take to get better performance results. (Idea: involve staff in designing the measures so they have more understanding and buy-in.)

These seven criteria are a good basis to judge your measures. You might like to make up a grid with your measures listed down the rows and these criteria listed across the columns. You can then do a quick evaluation of your existing measures, and see at a glance which to keep, which need some work, and which should probably been thrown away!

The debate shouldn’t, in my opinion, be about which people performance management approach is the correct one. It’s more about which approach achieves the intent you have for your organisation and your people, from the points of view of all stakeholders. And this means understanding the diversity of values that people in your organisation have, and the worldview that this collectively gives your organisation as a whole.

So you won’t likely find an approach that does work for your organisation, unless you can answer quite thoroughly several important questions:

• Why do you want to measure the performance of people?
• What is the end result you want it to create?
• What are the unintended consequences, or risks, of getting and having this end result?
• What other ways could you create this end?
• What worldview does your organisation have, and hence how is the relationship of people to the organisation understood?
• What values do the individual people in your organisation have, and what does this say about what will really motivate them (and keep them motivated)?

Read the full version of this article here: http://www.staceybarr.com/freeinfo/articles/PeopleMeasures.pdf

TAKE ACTION: Ask those questions above of the managers and HR professionals in your organisation, document their answers and contrast and compare their responses with each other, and with the current employee performance management approach you currently have.