Are Your Performance Measures Objective or Subjective?

August 22, 2017 by Stacey Barr

Does everyone agree with what your KPIs and performance measures are saying? If they’re arguing, objectivity could be the problem.

http://www.staceybarr.com/images/kaleandblueberries.jpg

Objectivity means that a measure is not biased or prejudiced by a person’s feelings or opinions or perceptions or mental filters. An objective measure is as close to fact as we can get. Objective measures are like kale and blueberries for our decision-making diet. Super nutritious.

But when measures lack objectivity, we call them subjective. This includes hearsay, opinion, data from very small samples (like a sample of one!), data from ambiguously asked questions, data from hand-picked samples, and assumptions or guesses. Subjective measures are like hamburgers and ice-cream for our decision-making. Junk food.

It does, of course, mean we need to value the health of our decision-making above its ease. It’s harder to choose kale over hamburgers. But if we do want decision-making health, we want our measures to have enough objectivity.

How to assess the objectivity of your measures:

In my years as a research statistician, we designed quantitative research around five essential qualities of measure integrity. A slightly adapted version of that list is useful when we want to take objectivity of our measures more seriously:

Relevant: An objective measure is direct evidence of the thing we’re measuring. When (and only when) the thing we’re measuring changes, the measure shows a correlated change. It’s why we measure how satisfied customers are using Average Customer Satisfaction Rating rather than Net Profit. Things other than customer satisfaction make profit change.

Representative: An objective measure is comprehensive enough to avoid bias or discrimination in the information it provides about what we’re measuring. It’s not showing part of the picture. It’s why we use random samples from the whole population, rather than volunteer samples or samples from a subpopulation (or, heaven forbid, from individuals), to collect our data from.

Reliable: An objective measure is based on enough data to discern signals from random variation. When the measure changes, we know it’s because the thing it measures has changed, and not because of overly sensitive data. A sample of 5 employees won’t give you a reliable measure of engagement, particularly if you have dozens or hundreds or thousands of employees.

Readable: An objective measure’s data is collected in a way that is clear and accurate, so errors are not introduced in its capture or analysis. If data can’t be read or understood, what ends up in datasets may not reflect the facts. Handwritten forms can often be impossible to decipher (as we find with our workshop feedback), so the important data for the measure is carefully structured to need only a hand-drawn circle.

Repeatable: An objective measure’s method of selecting, collecting, capturing and analysing its data can be performed independently by someone else, and they’d arrive at pretty much the same result. In PuMP we use a Measure Definition template (including basics like the formula) to make sure each measure is repeatable.

An objective measure is not biased or prejudiced by feelings, opinions, perceptions or mental filters.
[tweet this]

DISCUSSION:

Do you have a measure that everyone does trust, and is objective enough to be kale for your decision-making?

Facebooktwittergoogle_pluslinkedinmailby feather

Speak Your Mind

Your email address will not be published. Required fields are marked *

  1. Oly N says:

    Great blog post Stacey.

    I’ve been on a health journey for a couple of years and I can assure you that all the research & books you’ve been reading are right. Just as much as they’re wrong.

    The simple reason is that regarding health, we are all different. Our needs are slightly different in the first place depending our genetic imprint. And then there’s damage we’ve sustained as a result of how we live. Which is highly individual.

    So the art is to find what works for your individual situation.

    This is no different to the health of organisations.

    While there are some things that are important to every organisation as there is for health (e.g. smoking isn’t good for anybody), much depends on the individual organisation. What is right for one organisation might be entirely wrong for another.

    That’s why using indicator suites you can find on the internet don’t work. That’s why asking others to tell you what you should measure doesn’t work. That’s why brainstorming doesn’t work.

    For a healthy organisation and a healthy body, you need a purposeful targeted approach. I.e. define the goals that are important for you or your organisation, what needs to happen to achieve these, and then what interventions will get you there.

    Then constantly monitor the results and tweak the interventions if needed. And if they just don’t give you the results you’re after, scrap them and do more research again on the causes of your problem and ways to solve it.

    • Stacey Barr says:

      Oly, your extension of the metaphor of health into organisations is brilliant. Especially the bit about why indicator suites and brainstorming not working! Thanks for taking the time to share this. I think it will be a useful metaphor for others to use too.

Connect with Stacey


Haven’t found what you’re looking for? Want more information? Fill out the form below and I’ll get in touch with you as soon as possible.



*We respect your email privacy.
PO Box 422
Samford, Qld, 4520
Australia
Stacey Barr Pty Ltd
ACN: 129953635
Director: Stacey Barr
Simple Share Buttons