Why OKRs Do Not Help Us Measure What Matters

September 6, 2018 by Stacey Barr

Objectives and Key Results (OKRs) promise to help us measure what matters. But take a closer look, and you’ll see they have little to do with performance measurement.

Measure What Matters, by John Doerr

In the Kindle edition of his book Measure What Matters: OKRs – the Simple Idea That Grows 10x Growth, John Doerr celebrates the simple methodology that he originally learned from Intel legend, Andy Grove, in the 1970s. Doerr shares stories of how OKRs have helped many organisations, including Google, Bono and the Gates Foundation achieve great things.

Doerr gives this basic definition of an OKR, or Objective and Key Result, in his book:

“An OBJECTIVE… is simply WHAT is to be achieved, no more and no less. By definition, objectives are significant, concrete, action oriented, and (ideally) inspirational.”

“KEY RESULTS benchmark and monitor HOW we get to the objective. Effective KRs are specific and time-bound, aggressive yet realistic. Most of all, they are measurable and verifiable.”

Then Doerr shares this first example of an OKR, which was part of his presentation to introduce them to Google, to illustrate his definition. His example was an OKR he set for himself for that presentation:

OBJECTIVE:
Build a planning model for their company:

KEY RESULTS:
(AS MEASURED BY…)
KR #1: Finish the presentation on time.
KR #2: Create a sample set of quarterly Google OKRs.
KR #3: Gain management agreement for a three-month OKR trial.

What do OKRs have to do with measuring what matters?

OKRs are not the way to measure what matters.

In a book called Measure What Matters, I’d expect to find information about how to measure the things that matter, and I’d expect to find examples of meaningful measures. But I found neither.

The book has plenty of long stories about organisations and companies that have used OKRs, and does include a lot of examples of OKRs. But there is little instruction on measuring what matters, and plenty of examples of non-specific objectives, immeasurable key results, and key results that are actions rather than results. Like this one for Intel:

OBJECTIVE:
Demonstrate the 8080’s superior performance as compared to the Motorola 6800.

KEY RESULTS:
(AS MEASURED BY…)
1. Deliver five benchmarks.
2. Develop a demo.
3. Develop sales training materials for the field force.
4. Call three customers to prove the material works.

And this one, from MyFitnessPal:

OBJECTIVE:
Help more people around the world.

KEY RESULTS:
(AS MEASURED BY…)
1. Add 24M new users in 2014.
2. Reach 80M total registered users.

And just one more, from Intuit:

OBJECTIVE:
Deliver awesome end-to-end workforce technology solutions and strategies.

KEY RESULTS:
(AS MEASURED BY…)
1. Implement Box pilot for first 100 users by mid-quarter.
2. Complete BlueJeans rollout to final users by end of the quarter.
3. Transfer first 50 individual account Google users to enterprise account by end of the quarter.
4. Finalize Slack contract by end of month 1 and complete rollout play by end of the quarter.

These examples illustrate the very same problems we all face with how to measure our goals and strategy meaningfully: vague language, action-orientation, mistaking milestones for measurement, and hitting quotas rather than reaching for new and sustained levels of performance.

OKRs are not about measurable results, they are about implementation of actions.

I think what’s happening is that John Doerr and all the supporters of OKRs have different meanings for the words ‘result’ and ‘measure’. Their focus is strongly on implementation of action, not impact.

Every OKR example from the book listed actions instead of results. And it appeared (because it was never specifically articulated) that a measure was just the completion of those actions, or the meeting of a quota of actions.

Sure, there are different types of measures to monitor things. We can certainly use quotas and milestones to monitor progress, but we shouldn’t mistake them for performance measures. The OKR style of monitoring puts focus on implementing actions. Performance measures put focus on improving results. Most organisations already have too much focus on actions and not enough on results.

Why do great companies like Google say that OKRs work?

If OKRs don’t really help us measure what matters, why do these great companies love them so much? What does the OKR method contribute to organisational performance?

There are some common threads through each story in Measure What Matters. And these threads are, I think, what makes OKRs work:

  • Each organisation that loves OKRs appears to have a very inspirational and devoted leader, who believes in rallying the troops to work together.
  • Each leader who introduces OKRs believes in the power of focus, and ruthlessly prioritising what they give time and resources to.
  • The culture of those organisations that benefit from OKRs is one of getting things done.

So the point I really want to make is to consider OKRs as a tool for focused implementation of projects, but not as a tool for meaningful measurement.

If you want more meaningful performance measures, don’t mistake OKRs as a measurement tool.

If you want to have clear goals or objectives that people understand and that you can evidence the achievement of, then:

If you want meaningful measures that monitor results, then:

 

OKRs don’t help us measure what matters. They focus us on implementation, not results.
[tweet this]

DISCUSSION:

Does your organisation use OKRs? What success do you have with measuring in a meaningful way, to drive performance improvement? Or are you monitoring the progress of actions, rather than their results?

Facebooktwittergoogle_pluslinkedinmailFacebooktwittergoogle_pluslinkedinmail

Speak Your Mind

Your email address will not be published. Required fields are marked *

  1. Ghanshyam Pant says:

    You have echoed my reactions, Stacey!
    Many of the Measures examples you have quoted, are what we call as Strategic Actions or Projects that must be completed to achieve Strategic Goals.
    What we must measure is about what we do, our Processes. So it’s important to identify both the Effectiveness and Efficiency Measures of our Processes.

  2. Robert Scott says:

    Stacey, I agree totally with your comments, especially that OKR’s do not contribute to performance measurement in any useful way. I have read ‘Measure what matters’ – in fact I am presently reading it for the second time in an attempt to try and understand all the hype about OKR’s. I think its popularity stemmed from two things; its formulation by a high profile business leader, Andy Grove and supported by John Doerr, and its adoption by Google. It rapidly became the ‘in thing’ for business managers, a natural successor to MBO and others at a time when the tech industry wanted its very own business management tool. Although I agree that it doesn’t deal with performance measurement as is generally understood, it does satisfy the current desire to make all workers – certainly the managers and team leaders – feel part of the process of running the organisation, with OKR’s cascading down and each contributor being accountable in a very public way for his / her OKR’s. I feel it is more of a management / HR tool rather than one designed to measure progress towards stated goals. On the other hand, the more traditional approach to performance measurement may leave the decision making to a small group who then pass their decisions down the chain. I think OKR’s have value in their own way but I would certainly not use that approach to establish measurable goals, track the progress – or lack or – towards those goals or analyse and rectify variations beyond the norm.

    Regards, Robert

    • Stacey Barr says:

      Thanks for sharing your thoughts and insights, Rob. Regarding the cascading benefit of OKRs, I don’t think it’s unique to OKRs. In PuMP, for example, we use the Results Map to cascade goals in a collaborative and engaging way, and then those goals are measured by the people who own them, no matter what level they are in the organisation. They all get line of sight to the organisation’s purpose and direction. A client nicknamed it the “where’s Wally” effect of the Results Map: https://www.staceybarr.com/measure-up/a-measurable-strategy-on-a-single-page/

  3. Stacey, I picked up John Doerr’s book in a bookshop yesterday and quickly drew the same conclusion as you – OKRs might be used to assess completion of project activities, but the selected examples are far from a rigorous way of measuring “how well” an aspect of the business is performing. It’s a challenge to change such thinking when it’s coming from the organisation’s leader.Besides, who said we needed yet another measurement – related acronym?

    • Stacey Barr says:

      Exactly, Geoff – there is enough confusion about what “measure” and its related terms mean, and too many conflicting definitions. I think it’s the title of John Doerr’s book that has me the most miffed. It’s just not about measuring at all. Thanks for sharing your thoughts.

  4. Diana says:

    Thank you so much for this. I recently joined an organisation that uses OKRs and have been struggling to articulate why we need to do more to measure the right things. This really helps.

  5. Ed says:

    Hi. I work predominantly with entrepreneurial organisations. I am a huge fan of what you have done and I am also a huge fan of OKRs, in fact I combine them as follows. O’s for Moonshot thinking, stretch goals alla man on the moon and back safely in 10 years vs having to say communism sucks, us capitalism rules! Well written O’s create focus and drive, they excite and inspire.

    The KRs are the definition of done, simple as that. Has the goal got done, yes or no. However, I find that a company that uses your teachings is then able to include it in the KRs part of the OKR.

    I am also particular on wiping out anything that doesn’t add value to the products and services the businesses customers are paying for. Thus, every 90 days, processes owners using simple frameworks such as ‘making toast’ and leveraging your teachings can set OKRs that matter in the drive to continously simplify the business.

    So in short bringing them together you can combine moonshot thinking and measuring what matters but alla StaceyBarr!

    • Stacey Barr says:

      Ed, I agree and really love that you use PuMP-style measurement for the KRs. As long as we keep the measurement of results distinct from the monitoring of action milestones, but do them both for goals that really matter, we’re set for excellence.

  6. Mike Davidge says:

    Absolutely agree with you Stacey about OKRs based on what you have written (I haven’t read the book and now don’t intend to). The examples you quote are all focused on what the company needs to do and not on the customer. This is a basic mistake. We first need to understand what our customers want before considering how we deliver that. Toyota amongst others has understood this. So you are correct to insist on Results, that is aims that focus on the customer.

    • Stacey Barr says:

      Agreed, Mike. Customers – but also other stakeholder groups like the community, employees, etc… – definitely should be at the centre of planning. I don’t see OKRs as a planning process, that has the depth needed to think about an organisation’s ultimate purpose and responsibility to stakeholders.

Upcoming KPI Training


Our next PuMP Performance Measure Blueprint Workshops include:

>> London, UK, 5-6 Dec 2018

>> Ottawa, Canada, 26-27 Feb 2019

>> Brisbane, Australia, 13-15 Mar 2019

>> Melbourne, Australia, 15-17 May 2019

Register for the next PuMP Blueprint Workshop near you

Stacey's Books


Prove It! How to Create a High-Performance Culture and Measurable Success, book by Stacey Barr

Order Prove It! here.

Practical Performance Measurement: Using the PuMP Blueprint For Fast, Easy, And Engaging KPIs, book by Stacey Barr

Order Practical Performance Measurement here.

Reprinting Articles


You are welcome to use articles from the Measure Up blog, with these requirements

Connect with Stacey


Haven’t found what you’re looking for? Want more information? Fill out the form below and I’ll get in touch with you as soon as possible.



*We respect your email privacy.
Level 54, 111 Eagle Street
Brisbane, Qld, 4000
Australia
Stacey Barr Pty Ltd
ACN: 129953635
Director: Stacey Barr