Don’t Let Your Data Decide What You Measure

August 15, 2017 by Stacey Barr

Measuring performance takes some time and effort. But is that a good reason to rely only on readily-available data?

http://www.staceybarr.com/images/bigwaveandsurfer.jpg

Microsoft’s Workplace Analytics uses data that’s easy to capture: email subject lines, time stamps and calendar data from people’s use of their Microsoft applications. From this data, Workplace Analytics creates behavioral metrics such as these:

  • time spent in meetings
  • time spent collaborating
  • size of internal network
  • manager and employee one-on-one time
  • workload distribution
  • travel time to meetings
  • time worked after-hours

It’s too tempting to dive into analytics tools with ready-made measures.

Someone else has done the hard work. It’s now as easy as clicking buttons. This is the magnetic appeal of dashboards.

But if you dive in without a clear orientation, the deeper you dive the more likely you’ll lose your sense of where “up” is, and drown. Like a novice big-wave surfer being dumped by a 10m wave.

Microsoft say, on the back of Forrester research finding that employee productivity is the highest priority for C-level executives, that their new tool, Workplace Analytics, “provides unprecedented behavioral insights that can be used to improve productivity, workforce effectiveness and employee engagement”.

The hard work has been done for us, and the pretty screen-shots tempt us to dive right in. But the risk of drowning in all that data is very real.

When we aren’t clear what outcome matters to us, any measure will do.

And what the measure will do is waste our attention and energy on stuff that doesn’t make worthwhile differences. We dive into the data and get carried away in the current of clicking.

It might be interesting to know Network Size and Network Diversity for our workforce, to slice and dice to explore where and who and when. But to what purpose? Just because it’s available doesn’t mean it’s useful.

Even when we are clear what outcome matters to us, any measure put in front of us can blinker us.

Measures like Time Spent in Meetings and Time Spent Collaborating might sound like they’re telling us about employee productivity, but they’re not. Productivity is not just the input of where time is spent, but also the output created by how we spend that time.

We come to dashboards and analytics tools prematurely.

Data automation, dashboards and analytics tools are all great ideas. But we come to them too soon. They are almost always built on data that is readily or easily available. Limiting our performance measures to data that is available will make sure we never get the data we really need.

We need to come to data automation, dashboards and analytics tools after we’ve clearly defined the goals that matter, and designed the measures that give us direct evidence of those goals. We need to design these tools to answer the questions that matter, not give us answers to questions that are easily answered.

Limiting our KPIs to data that is available makes sure we never get the data that we need.
[tweet this]

DISCUSSION:

Do you have data in your dashboards or analytics tools that is there just because it’s available? How do you get the data you really need, when it’s not already available?

Facebooktwittergoogle_pluslinkedinmailby feather

Speak Your Mind

Your email address will not be published. Required fields are marked *

  1. Phil says:

    Hi Stacey, good timing. I am struggling with our reporting team who are promoting the use of new analytics technology in the BI space. The intent is fine as they are trying to promote people’s understanding of their own data but already we see local ‘measures’ popping up. Keeping a lid on the exhuberance with new technologies like this will be difficult without a performance measurement strategy.

    • Stacey Barr says:

      Agreed! Do you have a performance measurement strategy you could use, to add a front-end to the promoting process your team is using? Some tips for deciding what is worth measuring/knowing, and some tips for designing good measures? And tips for selecting diagnostic data to analyse causes in the measures? Then show how the tool makes it easier to bring them to life?

  2. Hi Stacey,

    Fully agree that you need to curate the data you need rather than only using what is readily available. To that extend how do you see covering relevant intangible measures/KPI’s here like employee satisfaction, engagement cooperation etc? Thanks and keep up the good work!

    • Stacey Barr says:

      Wouter, I don’t think there is such a thing as an intangible KPI. If you have a real, meaningful KPI then you’ve already made the attribute it measures tangible somehow.

      So sure, we have intangible goals. The first step is to make them tangible. And that means making them observable. Why have a goal if you can’t observe whether it’s happened or not? Search for ‘weasel words’ on this blog for ideas of how to do this.

      Second, we need to then decide how we’ll measure the now-tangible goal. And the key to this is quantifying the evidence we observe when the goal is achieved. Try this article: https://www.staceybarr.com/measure-up/five-steps-to-find-the-right-measures/

      The point: you can’t measure something intangible, only something that you can observe or detect in the real world.

  3. O'brain says:

    Hi Stacey,
    Thank you for this timely post. I am working on a project to develop a dashboard for data that is not available yet. Any tips on how to generate set of data for this project that will help me decide what to measure?

    • Stacey Barr says:

      I wouldn’t try to generate a set of data to decide what to measure. I think it’s more useful to it in this order:
      1) What is the purpose of the dashboard? Who is it for? What decisions will it inform? What goals are they trying to achieve? What questions should it answer?
      2) What measures will provide these answers?
      3) How should these measures be defined (quantified)?
      4) What data is required to calculate the measures’ values?

      That makes it sound easy, and it is a bit more involved. That’s why we have PuMP (http://staceybarr.com/about/pump).

  4. Tom G says:

    G’day friends. Thank you again for the encouragement to stay true to what is useful. These types of measures seem to come from the obsession on cost reduction. The focus on revenue and customer delight are so much more powerful.

Connect with Stacey


Haven’t found what you’re looking for? Want more information? Fill out the form below and I’ll get in touch with you as soon as possible.



*We respect your email privacy.
PO Box 422
Samford, Qld, 4520
Australia
Stacey Barr Pty Ltd
ACN: 129953635
Director: Stacey Barr
Simple Share Buttons