Can You Automate XmR Signal Rules in Your Dashboard?

by Stacey Barr

It’s tempting to have software automate the signals in our KPI XmR charts, but before you do, be aware of these risks and recommendations.

Robot hand and human hand putting puzzle pieces together. Credit: https://www.istockphoto.com/portfolio/AndreyPopov

XmR charts are my favourite chart for tracking performance measures. They give us a big enough picture to understand true changes in performance over time, and they give us valid signal rules to know if, when and by how much performance truly does change.

This article assumes you already know what XmR charts are, and how they work. But if XmR charts are a new concept to you, I recommend you read these articles first (they will be a revelation!):

A question I am asked a lot is whether the XmR chart signals can be automated. The short answer to this question is yes, they can. But a more important question to explore, before we go ahead and do it, is whether we should automate those signals.

My personal decision is that I won’t automate the signals. And XmR chart guru and author of “Understanding Variation: The Key to Managing Chaos”, Dr. Donald Wheeler, tells me he still takes the same position:

Process behavior charts are the interface between our data and our brains. Nothing is going to happen until our brains are engaged. Software can create the charts, but we should never depend upon the software to interpret the charts. The charts have to be interpreted in context, and the software never knows the context. — Dr Donald Wheeler

If you’re tempted to have software interpret your performance measures on your behalf, please read this article before you do!

How can XmR chart signals (and recalculations) be automated?

One reason why XmR charts can transform conversations about performance is that they make interpretation of our measures happen faster, with more accuracy and less disagreement. That’s because their signal rules are based in mathematical statistics and not opinion.

Because the signal rules are based in mathematical statistics, there is logic behind them. And where there is logic, there can usually be coding. Thusly, it’s a piece of cake for any programmer to set up the code that will look over a performance measure’s XmR chart, and detect those common signals.

The same goes for the recalculation of the Central Line and Natural Process Limits, in the case of a signal of change, because they too are each based on a specific formula.

Michel Dekker’s team, of Nova Silva in the Netherlands, has done just that, in the Microsoft Power BI XmR chart tool they have developed. Watch a short demonstration here:

 

 

The advantages of automated signals are easy to appreciate.

What are the advantages of automating XmR signals?

We generally automate things that are repetitive and rule-based, to free up our mental space for harder thinking tasks. That makes it only natural to see this same opportunity for XmR charts. When it comes to interpreting our performance measures, automating the signal rules can:

  • save us a minute or so of searching for the signals manually
  • alleviate the need for all users of the measures to learn the XmR chart theory
  • reduce our mistakes in applying incorrect rules, or applying rules incorrectly, to our measures
  • speed up the display of signals and associated traffic lights in our performance dashboards (since manual intervention isn’t require)

But to make the decision about whether to go with signal automation or not, we need to contrast these advantages with the risks of having software interpret our measures for us.

What are the risks of automating XmR signals?

The main reason for my hesitation, and Don Wheeler’s hesitation, to automate XmR signals is that software can never understand context. In fact, Don sums this up perfectly, as his first rule principle for understanding data:

“Data have no meaning apart from their context.” — Donald Wheeler, Understanding Variation [page 13]

Context matters in XmR chart interpretation. Here are several reasons why:

  • Due to random variation, a signal in an XmR chart might look like it starts before the cause of the signal actually happened in reality.
  • Some improvement initiatives take time to bed in, which we see as a gradual trend in the XmR chart. Automation will produce a series of short signals over the trend period, but the meaningful signal of change doesn’t really happen until after the bedding-in trend levels off.
  • If a performance measure is in chaos, there isn’t enough stability to apply XmR chart rules. But software has no way to detect chaos.

These reasons culminate into a significant risk of automating XmR chart signals: the software fails to match signals to real-world explanations.

I don’t expect you to adopt my approach to never automate XmR chart signal rules. But I do invite you to do is be careful how you use them.

How should we use automated XmR chart signals?

It takes a human eye plus knowledge of the system being measured to realise when an automated signal isn’t appropriate. If you do choose to use automated signals, this procedure will help to mitigate the risks and ensure you accurately interpret what performance is really doing:

  1. Decide if your performance measure is stable enough for an XmR chart, and if it’s not, focus on standardisation of the process being measured, until your measure shows stability.
  2. Look at the automated signal in your XmR chart, before any recalculations of the Central Line and Natural Process Limits are performed.
  3. Check if the signal makes sense, based on your knowledge of the real-world changes associated with that signal.
  4. If the automated signal isn’t starting at the right point in time to match the real-word change, correct it manually.
  5. Only after you’re happy that the signal of performance reflects the reality of performance, recalculate the Central Line and Natural Process Limits.

The only right thing for software to do with XmR charts is to give us the flexibility to adjust the signals and set when the recalculations should happen, if the automation isn’t appropriate. Michel Dekker and his Nova Silva team know this, and that’s why their Power BI XmR chart tool gives you complete control over signal detection and recalculation.

All new technology that replaces the more routine aspects of human processing can only do what we program it to do. And so far, BI software has no way of knowing the context upon which accurate interpretation of our data depends. And accurate interpretation is what good decision making requires.

Software automation of KPI signals is merely a starting point that requires our knowledge of context to draw correct conclusions.
[tweet this]

FacebooktwitterlinkedinmailFacebooktwitterlinkedinmail

Speak Your Mind

Your email address will not be published. Required fields are marked *

  1. Bill Wiest says:

    Hi Stacey,

    I’m curious to know, on the question of context, whether using a larger number of data points to calculate the Central Line gives you better context. If so, would you recommend, after arriving at the point where you have a larger number of data points (having started with 5), recalculating the Central Line with the larger number?

    • Stacey Barr says:

      Bill, no I don’t recommend that. Mostly on the advice of Donald Wheeler, that the Central Line doesn’t really get that much more accurate with more data points. But also because context is so much more than just historic data availability.

  2. Martien Dingemans says:

    Hi Stacey,

    I would like to react upon this article.

    Some years ago, we automated in our company the XmR in SQL server, for world wide support audience with many filter options.
    Providing all these filter options has the disadvantage that manual XmR creation is absolutely not an option any more, while the good thing is that these filters increases analytical options a lot.

    I agree that it is risky and that for an automated XmR should always be looked at the context, however in my experience that was the case for every report that we have created last 20 years.

    Key in performance management is in my opinion always to ‘Know what you look at’. If people do not know how an XmR works, they will definitely not know what actions they can take from it, manual or automated.
    In a company like ours, where we create statistics for hundreds of managers, it is most important to explain and provide good documentation, information sessions and even help them analyzing /drawing conclusion.
    That was valid for each and every report we provided, not only the XmR.
    I am pro automation, with the condition that like for each and every report, key is to make sure that the users are educated.
    (In a small company I agree that manual could be the best option)

    Regards,
    Martien.

    • Stacey Barr says:

      Thanks for sharing your story, Martien. A properly constructed and interpreted XmR chart that is automated is still better than most reports where users focus on comparing this month to last month, or this month to target.

Upcoming KPI Training


Our next PuMP Performance Measure Blueprint Workshops include:

>> North America, Online Interactive, 7-11 Feb 2022

>> Australia/NZ/Asia Pacific, Online Interactive, 23-24, 28 Feb, 1-2 Mar 2022

>> Africa, Online Interactive, 14-18 Mar 2022

>> Australia/NZ/Asia Pacific, Online Interactive, 29-31 Mar 2022

>> Africa, Online Interactive, 20-24 Jun 2022

>> Africa, Online Interactive, 26-30 Sep 2022

>> Africa, Online Interactive, 28 Nov - 2 Dec 2022

Register for the next PuMP Blueprint Workshop near you

Stacey's Books


Prove It! How to Create a High-Performance Culture and Measurable Success, book by Stacey Barr

Order Prove It! here.

Practical Performance Measurement: Using the PuMP Blueprint For Fast, Easy, And Engaging KPIs, book by Stacey Barr

Order Practical Performance Measurement here.

Reprinting Articles


You are welcome to use articles from the Measure Up blog, with these requirements

Connect with Stacey


Haven’t found what you’re looking for? Want more information? Fill out the form below and I’ll get in touch with you as soon as possible.



    *We respect your email privacy.
    Level 54, 111 Eagle Street
    Brisbane, Qld, 4000
    Australia
    Stacey Barr Pty Ltd
    ACN: 129953635
    Director: Stacey Barr