How to Improve KPI Accuracy and Precision

February 12, 2019 by Stacey Barr

Does your team spend more time debating the integrity of their KPIs than they do using them to improve performance?

image of darts hitting the bullseye

It always frustrates me how much time decision-makers spend arguing about the accuracy or precision or integrity of a measure, compared to the time they spend using that measure to inform decisions.

The terms accuracy and precision and integrity are often used interchangeably when we talk about KPIs. Yes, it’s important that our measures have enough of these things so that they don’t lead our decision-making astray. But how much KPI accuracy and precision and integrity do we need?

For a practical answer to this question, we need to first deweasel these terms: accuracy and precision and integrity.

Accuracy means your measure’s mean is close to the truth.

Imagine you’re shooting arrows at a classic bullseye target.

Accuracy is akin to how close, on average, your arrows will land relative to the centre of the bullseye. A measure is more accurate the more directly it is measuring the result you are trying to measure.

Average Customer Satisfaction Rating is a more accurate measure of customer satisfaction than Number of Customer Complaints is. That’s because customer complaints usually only come from the “squeaky wheels”, and they drown out the valid and important and balancing views of all other customers.

Squeaky wheels, volunteer surveys and easiest-ones-to-measure are examples of data sources that render your measures inaccurate. Your measures end up biased, which means they centre around a point that’s different to the bullseye.

Precision means your measure’s variation isn’t unnecessarily large.

Precision is akin to how close to each other your arrows are when they land on the target. A measure is more precise when it is based on enough data to get a reliable picture, as you measure it again and again over time.

Would you rely on one customer’s satisfaction rating to draw conclusions about all customers’ satisfaction? What about five customers? How many customers would you need to get a precise enough estimate of Average Customer Satisfaction Rating? There isn’t one right answer, because it does depend on your customer population and how variable their ratings are likely to be. But we can thank the survey statisticians for developing a method to calculate the best sample size in different situations.

Truth be told, KPIs or performance measures of any kind are never precise. Rather, they have a degree of precision that depends on the sample size for collecting the data, how that sample is selected, and how meticulously the data is collected and recorded. In deliberately designed data collection regimes, precision can be measured (e.g. using confidence intervals). If your measure’s data isn’t reliable, it means its calculations can vary wildly around the bullseye.

Minitab have an article that dives more deeply into accuracy and precision, with an excellent bullseye graphic to show their relationship to one another, and to the truth.

Integrity means your measure is accurate enough and precise enough for its purpose.

Integrity, in the context of KPIs, means your measures have enough accuracy and precision for your needs. And, from my background as research statistician, integrity can be unpacked into five actionable components that help us make practical decisions about getting enough accuracy and precision:

Integrity Dimensions Accuracy Precision
Relevant: The data selected is directly appropriate to the purpose of the performance measure it was selected it for. yes
Reliable: Enough data is collected to account for the inherent variability of what is being measured, over time. yes
Representative: The data describe the full scope of what your performance measure is supposed to be measuring, without bias. yes
Readable: The data is clearly defined, legibly presented, easy to organise for analysis, and makes sense to its users. yes yes
Realistic: Make sure the value you get from using your data is greater than the effort you invested in getting it. yes

Dive more deeply into the five “Rs” of data integrity here. And make sure you follow proven data collection design steps, to get the best data you can afford for each KPI.

How much accuracy and precision do your KPIs need?

A measure can be precise and accurate. But it can also be accurate but not precise, or precise but not accurate. Accuracy is mostly influenced by how well we design our
performance measure and it’s underlying data. Precision, however, is influenced by how much and how meticulously we collect a measure’s data.

If you ask me, I treat accuracy as more important than precision. Partly, because precision is more costly than accuracy. And mostly, because it doesn’t make sense to measure something inaccurately, with high precision. But we can still make sound decisions when we measure something accurately, with moderate precision.

Stop wasting time debating the integrity of your KPIs. Make them just accurate enough, and use them to improve performance.
[tweet this]

DISCUSSION:

Do you have the problem of people debating KPI integrity, and never getting to the point where the KPI is used to improve performance?

Facebooktwittergoogle_pluslinkedinmailFacebooktwittergoogle_pluslinkedinmail

Speak Your Mind

Your email address will not be published. Required fields are marked *

    Upcoming KPI Training


    Our next PuMP Performance Measure Blueprint Workshops include:

    >> London, United Kingdom, 1-2 May 2019

    >> Nairobi, Kenya, 8-10 May 2019

    >> Melbourne, Australia, 15-17 May 2019

    >> Calgary, Canada, 3-5 Jun 2019

    >> Accra, Ghana, 10-12 Jul 2019

    >> London, United Kingdom, 11-12 Sep 2019

    >> Nairobi, Kenya, 11-13 Sep 2019

    >> Johannesburg, South Africa, 12-15 Nov 2019

    >> London, United Kingdom, 3-4 Dec 2019

    Register for the next PuMP Blueprint Workshop near you

    Stacey's Books


    Prove It! How to Create a High-Performance Culture and Measurable Success, book by Stacey Barr

    Order Prove It! here.

    Practical Performance Measurement: Using the PuMP Blueprint For Fast, Easy, And Engaging KPIs, book by Stacey Barr

    Order Practical Performance Measurement here.

    Reprinting Articles


    You are welcome to use articles from the Measure Up blog, with these requirements

    Connect with Stacey


    Haven’t found what you’re looking for? Want more information? Fill out the form below and I’ll get in touch with you as soon as possible.



    *We respect your email privacy.
    Level 54, 111 Eagle Street
    Brisbane, Qld, 4000
    Australia
    Stacey Barr Pty Ltd
    ACN: 129953635
    Director: Stacey Barr