How to Set a KPI Baseline to Monitor Improvement

January 10, 2018 by Stacey Barr

We can’t know if performance improves unless we know where it started. That’s why we set a KPI baseline. Here’s how to set your KPI baseline the right way!

http://www.staceybarr.com/images/feetagainstaline.jpg

Because most people lack an understanding of natural variability, they will set baselines for their KPIs the wrong way:

  • average of last 12 months, which is wrong because over a 12 month period, there can be many shifts and changes in performance that are different to where performance is now
  • last month’s performance, which is wrong because a single performance measure value is too subject to random variability to be representative of the KPI’s true level of performance
  • using the target for the KPI, which is wrong because the target is where we want performance to be, but a baseline is where performance is now

Everything varies, including performance. The trick to setting a sensible and useful KPI baseline is to use enough performance measure values to calculate it, and not use any more than you need. The minimum number of performance measure values you’ll need is 5. Sometimes you need more than that, if the KPI’s variation is a little chaotic.

And the method to set your KPI baseline will depend on the maturity of you KPI. In some cases you can use historic data to set the baseline. But in other cases, you might need to collect some data for a while before you can set it.

Let’s explore how to set a KPI baseline for three common scenarios:

  1. a brand-spanking new KPI with no data yet
  2. a mature KPI with a lot of historical data
  3. a mature KPI with a seasonal or cyclical pattern

If you have a brand-spanking new KPI with no data yet…

A new KPI for my business is PuMP Pilot Started, and it’s a simple count of the number of members in our PuMP Community that have started to implement PuMP after their training, on a simple pilot project.

I’ve only just finished setting up the new PuMP Community with a dashboard for members to log their PuMP Pilot progress, so I don’t really have much data yet to set a baseline.

At the moment, I have a spreadsheet set up, where I’m logging each member and the date on which they started their PuMP Pilot (data emailed to me from the PuMP Community dashboard).

But because I don’t really have enough data yet, I don’t know whether to measure this weekly or monthly. I’d want several pilots started each week, to make it worth measuring weekly. Then when I have five week’s of data, I will average the weekly totals to create the baseline.

TIP: Don’t create a baseline until you have a good feel for the frequency of calculation for your measure, and you have enough data for five measure values. Then average those measure values to set your baseline. You need at least five measure values for a valid baseline.

If you have a mature KPI with a lot of historical data…

The PuMP Blueprint Workshop has been running since 2006, but in 2007 I started to measure it’s Net Promoter Score, or NPS. Even if I focus just on the public workshops, there is a lot of historical data for this measure now.

http://www.staceybarr.com/images/pumpblueprintworkshopnetpromoterscore.jpg

If I hadn’t done it already (and of course, I have), I’d set up a baseline for the PuMP workshop’s NPS by averaging the first five values in my time series. Sure, that was years ago, but you can see how that historic baseline immediately highlights a subsequent improvement in 2010 (using the signal interpretation of XmR charts):

http://www.staceybarr.com/images/pumpblueprintworkshopnetpromoterscorebaseline.jpg

So a new baseline can be set from the start of that improvement, and it helps to see future changes. Can you see the new change in NPS after updating the baseline?

http://www.staceybarr.com/images/pumpblueprintworkshopnetpromoterscoreimprovement.jpg

I don’t know the cause the for dip in performance that started in late 2013. I’m not worried because it was temporary, and also because an NPS above 50% is still considered excellent. (That doesn’t mean I’m not still working to improve NPS – but only when it’s a priority compared with improving other KPIs.)

TIP: Don’t be afraid to create historic baselines. When you do have a lot of historical data, you can start your baseline in the past, and use it to retrospectively see how performance has changed since that point in time.

If you have a mature KPI with a seasonal or cyclical pattern…

Seasonal patterns in our KPIs can be caused by lots of factors other than Mother Nature, like school holidays, financial or management reporting rhythms, shift work or maintenance schedules.

This KPI is Peak Day Water Usage, and clearly it has a seasonal pattern:

http://www.staceybarr.com/images/peakdaywaterusageseasonal.jpg

Before you set a baseline for a KPI that has a seasonal pattern, you must de-seasonalise it! Otherwise your baseline won’t be useful for any other part of the seasonal cycle than the one it was calculated in. Here is Peak Day Water Usage in its de-seasonalised form:

http://www.staceybarr.com/images/peakdaywaterusagedeseasonalised.jpg

After we de-seasonlise Peak Day Water Usage, the current baseline can now be set by taking the average of the first 5 points that define our starting performance level:

http://www.staceybarr.com/images/peakdaywaterusagebaseline.jpg

And now you can see, quantitatively by the difference in the baselines, how much this KPI improved in mid-2009:

http://www.staceybarr.com/images/peakdaywaterusageimprovement.jpg

TIP: It makes no sense to set a baseline for a seasonal KPI, because you’ll be at risk of mistaking a seasonal change for a real change in performance. It’s easy to de-seasonalise your KPI before you set a baseline.

Looking for evidence of improvement from the baseline?

In my world, a baseline is simply a quantitative value that describes current performance. When performance changes, for good or bad, the baseline changes. Your definition might be a bit different, and it doesn’t matter. What matters is that we understand, objectively and quantitatively, where performance was in the past, where it is now, and do the improvement projects that will shift it to where we want to be.

We can’t know if performance improves unless we know where it started. That’s why we set a KPI baseline.
[tweet this]

DISCUSSION:

How are KPI baselines set in your organisation? Are they sensible and helpful in driving performance improvement?

Facebooktwittergoogle_pluslinkedinmailFacebooktwittergoogle_pluslinkedinmail

Speak Your Mind

Your email address will not be published. Required fields are marked *

  1. Martin Andrew says:

    Stacey, for baseline for a new KPI, why not set it based on what level of performance the organisation requires, or expects the new process to deliver? I.e. use a Planned approach.

    • Stacey Barr says:

      Martin, there is a distinction between where performance actually is, and where we want it to be. The former is the baseline and the latter is the target. We have the KPI most likely because it is something we want to improve, which means we’re not at the level of performance the organisation requires just yet. We need improvement projects to shift to that level. When we reach it, it means our baseline and our target will be the same level.

  2. calvin says:

    Are there situations where there is simply too much natural variation that it becomes extremely difficult to de-seasonalize a KPI or establish a baseline? Where the amount of data needed to understand the variation is too complex? If so, are we left with measuring completion of actions or effort, as opposed to measuring results?

    • Stacey Barr says:

      When there is too much variation, to me that means performance is chaotic. Whatever process or system is producing the result you’re measuring, it has little standardisation or control to guide it. For example, it’s done inconsistently all the time, and that’s what is creating the excessive variation. Keep measuring, but focus on reducing the amount of variation. In XmR charts, that would be the width of the natural process limits calculated from the data. Reduce the variation by standardising the steps of the process, or the ways that things are done. Try to get some more consistency. Then you will see the variation come down.

Connect with Stacey


Haven’t found what you’re looking for? Want more information? Fill out the form below and I’ll get in touch with you as soon as possible.



*We respect your email privacy.
Level 54, 111 Eagle Street
Brisbane, Qld, 4000
Australia
Stacey Barr Pty Ltd
ACN: 129953635
Director: Stacey Barr