Does everyone agree with what your KPIs and performance measures are saying? If they’re arguing, objectivity could be the problem.

Objectivity means that a measure is not biased or prejudiced by a person’s feelings or opinions or perceptions or mental filters. An objective measure is as close to fact as we can get. Objective measures are like kale and blueberries for our decision-making diet. Super nutritious.

But when measures lack objectivity, we call them subjective. This includes hearsay, opinion, data from very small samples (like a sample of one!), data from ambiguously asked questions, data from hand-picked samples, and assumptions or guesses. Subjective measures are like hamburgers and ice-cream for our decision-making. Junk food.

It does, of course, mean we need to value the health of our decision-making above its ease. It’s harder to choose kale over hamburgers. But if we do want decision-making health, we want our measures to have enough objectivity.

How to assess the objectivity of your measures:

In my years as a research statistician, we designed quantitative research around five essential qualities of measure integrity. A slightly adapted version of that list is useful when we want to take objectivity of our measures more seriously:

Relevant: An objective measure is direct evidence of the thing we’re measuring. When (and only when) the thing we’re measuring changes, the measure shows a correlated change. It’s why we measure how satisfied customers are using Average Customer Satisfaction Rating rather than Net Profit. Things other than customer satisfaction make profit change.

Representative: An objective measure is comprehensive enough to avoid bias or discrimination in the information it provides about what we’re measuring. It’s not showing part of the picture. It’s why we use random samples from the whole population, rather than volunteer samples or samples from a subpopulation (or, heaven forbid, from individuals), to collect our data from.

Reliable: An objective measure is based on enough data to discern signals from random variation. When the measure changes, we know it’s because the thing it measures has changed, and not because of overly sensitive data. A sample of 5 employees won’t give you a reliable measure of engagement, particularly if you have dozens or hundreds or thousands of employees.

Readable: An objective measure’s data is collected in a way that is clear and accurate, so errors are not introduced in its capture or analysis. If data can’t be read or understood, what ends up in datasets may not reflect the facts. Handwritten forms can often be impossible to decipher (as we find with our workshop feedback), so the important data for the measure is carefully structured to need only a hand-drawn circle.

Repeatable: An objective measure’s method of selecting, collecting, capturing and analysing its data can be performed independently by someone else, and they’d arrive at pretty much the same result. In PuMP we use a Measure Definition template (including basics like the formula) to make sure each measure is repeatable.

An objective measure is not biased or prejudiced by feelings, opinions, perceptions or mental filters.
[tweet this]

DISCUSSION:

Do you have a measure that everyone does trust, and is objective enough to be kale for your decision-making?