In God We Trust, All Others Bring Data

Deming’s famous t-shirt quote, in God we trust, all others must bring data, is extreme. Is it too extreme?

One of my recently discovered favourite quotes about measurement is this:

One measurement is worth a thousand expert opinions. — Donald Sutherland

It’s a lot like a much more famous quote (which I had on a t-shirt once):

In God we trust; all others must bring data. — W. Edwards Deming

One of the primary reasons I cannot watch the media for news is that it is rife with opinions. Even when they use data, they manipulate it to support the opinions. And, over time, the opinions regularly contradict each other.

The media is not a vehicle for the whole truth. It’s not even a vehicle for enough of the truth to support reliable conclusions.

Opinions might be fast and easy and more emotive than data, but rarely are they anywhere near as close to the truth as good data. Opinions save time and effort early, but cost many more times their worth later. Data might cost time and effort early, but it returns many more times its worth later.

But that’s not to say that experts cannot bring wisdom to a situation. Data is useful only when we ask the right questions. And experts are those with enough experience to know the right questions. They are very important leaders of the acquisition and interpretation of the right data.

We could augment what Deming said to be more like “in credible experts we trust; all others must bring data”. But I prefer to say something like this:

When we ask the right questions about the results that matter, we can more easily find the right data and information for evidence-based decisions that improve those results faster and with less effort. Opinion-led decisions take much longer and much more effort, often for much less (if any) improvement.

Opinions might be fast and easy and more emotive than data, but rarely are they anywhere near as close to the truth as good data.. [tweet this]

1. Bob McGlynn says:

   January 5, 2022 at 1:15 am

   You write that people with strong opinions (concerning issues in the news) use data to support their opinions. How can we ensure that when we create measure that matter it’s not just an opinion we are looking to support with data? For those with opinions (in the news) definitely feel their measures are meaningful.

   Reply
   • Stacey Barr says:

     January 27, 2022 at 8:55 am

     A few things help to avoid selecting measures that support opinions:

     1. Design measures always in teams, with team members that comprise a mix of people that understand the system to be measured from different angles. In PuMP we have a recipe for putting together a Measures Team.

     2. Don’t start with measure design, but start with a clear articulation of the results that matter (and I don’t mean the result you want the measure to support – I mean describe the outcome or impact or state that you want to understand, monitor and improve). In PuMP we have Step 2 that helps us write specific and clear and measurable results.

     3. When you come to measure design, do it objectively and start by considering what is the most convincing, observable evidence of the result or impact or outcome. Evaluate each potential measure deliberately, again as a team. Step 3 in PuMP does this.

     4. When you have results and measures drafted, open up the process to other stakeholders for their questions, ideas and feedback. Step 4 in PuMP does this.

     All those 4 steps before locking in on any measure.

     Reply