Two Principles For Reporting Your Performance Measures

by Stacey Barr |

We slice, we dice, we tabulate, we chart, we dashboard and then we draw conclusions. But how often have we drawn diabolically incorrect conclusions because we have violated the two basic principles of understanding our measures?

In Donald Wheeler’s book Understanding Variation, he introduces two very simple and sensible principles for understanding data:

  1. No data have meaning apart from their context.
  2. While every data set contains noise, some data sets contain signals. Therefore, before you can detect a signal within any given data set, you must first filter out the noise.

What do these principles mean, particularly in the context of reporting performance measures or KPIs?

PRINCIPLE 1: No data have meaning apart from their context.

The first principle means that you must understand where your data come from, and you must have enough data, before you can understand what a measure is telling you.

Here are some practical tips to ensure you report your measures, meaningfully, with enough context:

  • Provide a link to and a summary of the meaning of the measure: it’s calculation, data source and plain-english description of what the measure means.
  • Report more than just one or two values of your measure. In fact, do your best to report your measures in time series, including up to 3 years’ worth of historical values.
  • Put your measures into graphs so it’s faster and easier to see patterns over time. When we’re trying to improve performance, it’s the patterns over time that tell the most insightful stories of performance.

PRINCIPLE 2: Before you can detect a signal, you must first filter out the noise.

The second principle means that every time you compute a difference between two values, you cannot assume that difference means there was a change. These computations include this month to last month, this month to the same month last year, this month compared to target, and so on.

We all know how to quantify performance using averages and totals and percentages. But in order to filter out the noise in our measures, we also need to quantify the variation.

Here are some practical tips to ensure you report real signals in your measures (and not be misled by the noise):

  • For performance measures you monitor regularly over time, use XmR charts.
  • When you are reporting survey results that were based on a sample, report the confidence intervals for the survey estimates.

I’m really looking forward to data visualisation expert Stephen Few’s next book on this very topic, called Signal.


Take a look at your performance reports and see if you can find any measure or KPI presentation that is violating the two principles for understanding data. What risks does this expose your decision-making to?

Speak Your Mind

Your email address will not be published. Required fields are marked *

  1. Bob Frost says:


    Thanks for reminding us of these two key points. We constantly see people violating the second principle–assuming that any change between two points in the data is actually real and meaningful.

    And thanks for the alert that Stephen Few has a new book coming; I’m sure it will be as great as his others.

    Bob Frost

    • Stacey Barr says:

      Hi Bob, thanks for dropping by. You’re a man after my own heart: I agree about how many people violate the second principle.

      My motto is: patterns, not points. And XmR charts make the patterns (the real ones) so very obvious.

Upcoming KPI Training

>> North America, Online Interactive, 6-10 February 2023 - SOLD OUT

>> Africa, Online Interactive, 27 February - 3 March 2023

>> UK & Europe, Online Interactive, 27 February - 3 March 2023

>> North America, In-Person, Calgary AB, 7-9 March 2023

>> Australia/NZ/Asia Pacific, Online Interactive, 15-16 & 20-22 March 2023

>> Africa, In-Person, Dubai UAE, 3-5 May 2023

>> Australia/NZ/Asia Pacific, In-Person, Wellington NZ, 9-11 May 2023

>> UK & Europe, Online Interactive, 22-26 May 2023

>> North America, Online Interactive, 29 May - 2 June 2023

Register for the next PuMP Blueprint Workshop near you

Reprinting Articles

You are welcome to use articles from the Measure Up blog, with these requirements

Connect with Stacey

Haven’t found what you’re looking for? Want more information? Fill out the form below and I’ll get in touch with you as soon as possible.

    *We respect your email privacy.
    Suite 117 Level 14,
    167 Eagle Street,
    Brisbane Qld 4000,
    Stacey Barr Pty Ltd
    ACN: 129953635
    Director: Stacey Barr