Two Principles For Reporting Your Performance Measures

by Stacey Barr |

We slice, we dice, we tabulate, we chart, we dashboard and then we draw conclusions. But how often have we drawn diabolically incorrect conclusions because we have violated the two basic principles of understanding our measures?

In Donald Wheeler’s book Understanding Variation, he introduces two very simple and sensible principles for understanding data:

  1. No data have meaning apart from their context.
  2. While every data set contains noise, some data sets contain signals. Therefore, before you can detect a signal within any given data set, you must first filter out the noise.

What do these principles mean, particularly in the context of reporting performance measures or KPIs?

PRINCIPLE 1: No data have meaning apart from their context.

The first principle means that you must understand where your data come from, and you must have enough data, before you can understand what a measure is telling you.

Here are some practical tips to ensure you report your measures, meaningfully, with enough context:

  • Provide a link to and a summary of the meaning of the measure: it’s calculation, data source and plain-english description of what the measure means.
  • Report more than just one or two values of your measure. In fact, do your best to report your measures in time series, including up to 3 years’ worth of historical values.
  • Put your measures into graphs so it’s faster and easier to see patterns over time. When we’re trying to improve performance, it’s the patterns over time that tell the most insightful stories of performance.

PRINCIPLE 2: Before you can detect a signal, you must first filter out the noise.

The second principle means that every time you compute a difference between two values, you cannot assume that difference means there was a change. These computations include this month to last month, this month to the same month last year, this month compared to target, and so on.

We all know how to quantify performance using averages and totals and percentages. But in order to filter out the noise in our measures, we also need to quantify the variation.

Here are some practical tips to ensure you report real signals in your measures (and not be misled by the noise):

  • For performance measures you monitor regularly over time, use XmR charts.
  • When you are reporting survey results that were based on a sample, report the confidence intervals for the survey estimates.

I’m really looking forward to data visualisation expert Stephen Few’s next book on this very topic, called Signal.


Take a look at your performance reports and see if you can find any measure or KPI presentation that is violating the two principles for understanding data. What risks does this expose your decision-making to?


Speak Your Mind

Your email address will not be published. Required fields are marked *

  1. Bob Frost says:


    Thanks for reminding us of these two key points. We constantly see people violating the second principle–assuming that any change between two points in the data is actually real and meaningful.

    And thanks for the alert that Stephen Few has a new book coming; I’m sure it will be as great as his others.

    Bob Frost

    • Stacey Barr says:

      Hi Bob, thanks for dropping by. You’re a man after my own heart: I agree about how many people violate the second principle.

      My motto is: patterns, not points. And XmR charts make the patterns (the real ones) so very obvious.

Upcoming KPI Training

Our next PuMP Performance Measure Blueprint Workshops include:

>> United Kingdom, Online Interactive 5, 12, 19, 26 Nov & 3 Dec 2021

>> Australia/NZ/Asia Pacific, Online Interactive, 10-11, 15-17 Nov 2021

>> North America, Online Interactive, 15-19 Nov 2021

>> Africa, Online Interactive, 14-18 Mar 2022

>> Africa, Online Interactive, 20-24 Jun 2022

>> Africa, Online Interactive, 26-30 Sep 2022

>> Africa, Online Interactive, 28 Nov - 2 Dec 2022

Register for the next PuMP Blueprint Workshop near you

Stacey's Books

Prove It! How to Create a High-Performance Culture and Measurable Success, book by Stacey Barr

Order Prove It! here.

Practical Performance Measurement: Using the PuMP Blueprint For Fast, Easy, And Engaging KPIs, book by Stacey Barr

Order Practical Performance Measurement here.

Reprinting Articles

You are welcome to use articles from the Measure Up blog, with these requirements

Connect with Stacey

Haven’t found what you’re looking for? Want more information? Fill out the form below and I’ll get in touch with you as soon as possible.

    *We respect your email privacy.
    Level 54, 111 Eagle Street
    Brisbane, Qld, 4000
    Stacey Barr Pty Ltd
    ACN: 129953635
    Director: Stacey Barr