Why OKRs Do Not Help Us Measure What Matters
by Stacey Barr |Objectives and Key Results (OKRs) promise to help us measure what matters. But take a closer look, and you’ll see they have little to do with performance measurement.
In the Kindle edition of his book Measure What Matters: OKRs – the Simple Idea That Grows 10x Growth, John Doerr celebrates the simple methodology that he originally learned from Intel legend, Andy Grove, in the 1970s. Doerr shares stories of how OKRs have helped many organisations, including Google, Bono and the Gates Foundation achieve great things.
Doerr gives this basic definition of an OKR, or Objective and Key Result, in his book:
“An OBJECTIVE… is simply WHAT is to be achieved, no more and no less. By definition, objectives are significant, concrete, action oriented, and (ideally) inspirational.”
“KEY RESULTS benchmark and monitor HOW we get to the objective. Effective KRs are specific and time-bound, aggressive yet realistic. Most of all, they are measurable and verifiable.”
Then Doerr shares this first example of an OKR, which was part of his presentation to introduce them to Google, to illustrate his definition. His example was an OKR he set for himself for that presentation:
OBJECTIVE:
Build a planning model for their company:
KEY RESULTS:
(AS MEASURED BY…)
KR #1: Finish the presentation on time.
KR #2: Create a sample set of quarterly Google OKRs.
KR #3: Gain management agreement for a three-month OKR trial.
What do OKRs have to do with measuring what matters?
OKRs are not the way to measure what matters.
In a book called Measure What Matters, I’d expect to find information about how to measure the things that matter, and I’d expect to find examples of meaningful measures. But I found neither.
The book has plenty of long stories about organisations and companies that have used OKRs, and does include a lot of examples of OKRs. But there is little instruction on measuring what matters, and plenty of examples of non-specific objectives, immeasurable key results, and key results that are actions rather than results. Like this one for Intel:
OBJECTIVE:
Demonstrate the 8080’s superior performance as compared to the Motorola 6800.
KEY RESULTS:
(AS MEASURED BY…)
1. Deliver five benchmarks.
2. Develop a demo.
3. Develop sales training materials for the field force.
4. Call three customers to prove the material works.
And this one, from MyFitnessPal:
OBJECTIVE:
Help more people around the world.
KEY RESULTS:
(AS MEASURED BY…)
1. Add 24M new users in 2014.
2. Reach 80M total registered users.
And just one more, from Intuit:
OBJECTIVE:
Deliver awesome end-to-end workforce technology solutions and strategies.
KEY RESULTS:
(AS MEASURED BY…)
1. Implement Box pilot for first 100 users by mid-quarter.
2. Complete BlueJeans rollout to final users by end of the quarter.
3. Transfer first 50 individual account Google users to enterprise account by end of the quarter.
4. Finalize Slack contract by end of month 1 and complete rollout play by end of the quarter.
These examples illustrate the very same problems we all face with how to measure our goals and strategy meaningfully: vague language, action-orientation, mistaking milestones for measurement, and hitting quotas rather than reaching for new and sustained levels of performance.
OKRs are not about measurable results, they are about implementation of actions.
I think what’s happening is that John Doerr and all the supporters of OKRs have different meanings for the words ‘result’ and ‘measure’. Their focus is strongly on implementation of action, not impact.
Every OKR example from the book listed actions instead of results. And it appeared (because it was never specifically articulated) that a measure was just the completion of those actions, or the meeting of a quota of actions.
Sure, there are different types of measures to monitor things. We can certainly use quotas and milestones to monitor progress, but we shouldn’t mistake them for performance measures. The OKR style of monitoring puts focus on implementing actions. Performance measures put focus on improving results. Most organisations already have too much focus on actions and not enough on results.
Why do great companies like Google say that OKRs work?
If OKRs don’t really help us measure what matters, why do these great companies love them so much? What does the OKR method contribute to organisational performance?
There are some common threads through each story in Measure What Matters. And these threads are, I think, what makes OKRs work:
- Each organisation that loves OKRs appears to have a very inspirational and devoted leader, who believes in rallying the troops to work together.
- Each leader who introduces OKRs believes in the power of focus, and ruthlessly prioritising what they give time and resources to.
- The culture of those organisations that benefit from OKRs is one of getting things done.
So the point I really want to make is to consider OKRs as a tool for focused implementation of projects, but not as a tool for meaningful measurement.
If you want more meaningful performance measures, don’t mistake OKRs as a measurement tool.
If you want to have clear goals or objectives that people understand and that you can evidence the achievement of, then:
- write them well, as results and without weasel words
- cascade them through cause-effect logic
If you want meaningful measures that monitor results, then:
- understand what performance measures really are
- design them deliberately rather than brainstorming or defaulting to what’s easy
- monitor them for signals of change over time, in response to your actions
OKRs don’t help us measure what matters. They focus us on implementation, not results.
[tweet this]
DISCUSSION:
Does your organisation use OKRs? What success do you have with measuring in a meaningful way, to drive performance improvement? Or are you monitoring the progress of actions, rather than their results?
Speak Your Mind
Join Measure Up
Sign up for the Measure Up newsletter and get free access to the "10 Secrets to KPI Success" online course and e-book.
Upcoming KPI Training
>> Africa, In-Person, Dubai UAE, 3-5 May 2023
>> Australia/NZ/Asia Pacific, In-Person, Wellington NZ, 9-11 May 2023
>> UK & Europe, Online Interactive, 22-26 May 2023
>> North America, Online Interactive, 29 May - 2 June 2023
Connect with Stacey
Haven’t found what you’re looking for? Want more information? Fill out the form below and I’ll get in touch with you as soon as possible.
167 Eagle Street,
Brisbane Qld 4000,
Australia
ACN: 129953635
Director: Stacey Barr
You have echoed my reactions, Stacey!
Many of the Measures examples you have quoted, are what we call as Strategic Actions or Projects that must be completed to achieve Strategic Goals.
What we must measure is about what we do, our Processes. So it’s important to identify both the Effectiveness and Efficiency Measures of our Processes.
Thanks for sharing your thoughts, Ghanshyam.
Stacey, I agree totally with your comments, especially that OKR’s do not contribute to performance measurement in any useful way. I have read ‘Measure what matters’ – in fact I am presently reading it for the second time in an attempt to try and understand all the hype about OKR’s. I think its popularity stemmed from two things; its formulation by a high profile business leader, Andy Grove and supported by John Doerr, and its adoption by Google. It rapidly became the ‘in thing’ for business managers, a natural successor to MBO and others at a time when the tech industry wanted its very own business management tool. Although I agree that it doesn’t deal with performance measurement as is generally understood, it does satisfy the current desire to make all workers – certainly the managers and team leaders – feel part of the process of running the organisation, with OKR’s cascading down and each contributor being accountable in a very public way for his / her OKR’s. I feel it is more of a management / HR tool rather than one designed to measure progress towards stated goals. On the other hand, the more traditional approach to performance measurement may leave the decision making to a small group who then pass their decisions down the chain. I think OKR’s have value in their own way but I would certainly not use that approach to establish measurable goals, track the progress – or lack or – towards those goals or analyse and rectify variations beyond the norm.
Regards, Robert
Thanks for sharing your thoughts and insights, Rob. Regarding the cascading benefit of OKRs, I don’t think it’s unique to OKRs. In PuMP, for example, we use the Results Map to cascade goals in a collaborative and engaging way, and then those goals are measured by the people who own them, no matter what level they are in the organisation. They all get line of sight to the organisation’s purpose and direction. A client nicknamed it the “where’s Wally” effect of the Results Map: https://www.staceybarr.com/measure-up/a-measurable-strategy-on-a-single-page/
Stacey, I picked up John Doerr’s book in a bookshop yesterday and quickly drew the same conclusion as you – OKRs might be used to assess completion of project activities, but the selected examples are far from a rigorous way of measuring “how well” an aspect of the business is performing. It’s a challenge to change such thinking when it’s coming from the organisation’s leader.Besides, who said we needed yet another measurement – related acronym?
Exactly, Geoff – there is enough confusion about what “measure” and its related terms mean, and too many conflicting definitions. I think it’s the title of John Doerr’s book that has me the most miffed. It’s just not about measuring at all. Thanks for sharing your thoughts.
Thank you so much for this. I recently joined an organisation that uses OKRs and have been struggling to articulate why we need to do more to measure the right things. This really helps.
Glad this helps you, Diana. Thanks for sharing your thoughts.
Hi. I work predominantly with entrepreneurial organisations. I am a huge fan of what you have done and I am also a huge fan of OKRs, in fact I combine them as follows. O’s for Moonshot thinking, stretch goals alla man on the moon and back safely in 10 years vs having to say communism sucks, us capitalism rules! Well written O’s create focus and drive, they excite and inspire.
The KRs are the definition of done, simple as that. Has the goal got done, yes or no. However, I find that a company that uses your teachings is then able to include it in the KRs part of the OKR.
I am also particular on wiping out anything that doesn’t add value to the products and services the businesses customers are paying for. Thus, every 90 days, processes owners using simple frameworks such as ‘making toast’ and leveraging your teachings can set OKRs that matter in the drive to continously simplify the business.
So in short bringing them together you can combine moonshot thinking and measuring what matters but alla StaceyBarr!
Ed, I agree and really love that you use PuMP-style measurement for the KRs. As long as we keep the measurement of results distinct from the monitoring of action milestones, but do them both for goals that really matter, we’re set for excellence.
Absolutely agree with you Stacey about OKRs based on what you have written (I haven’t read the book and now don’t intend to). The examples you quote are all focused on what the company needs to do and not on the customer. This is a basic mistake. We first need to understand what our customers want before considering how we deliver that. Toyota amongst others has understood this. So you are correct to insist on Results, that is aims that focus on the customer.
Agreed, Mike. Customers – but also other stakeholder groups like the community, employees, etc… – definitely should be at the centre of planning. I don’t see OKRs as a planning process, that has the depth needed to think about an organisation’s ultimate purpose and responsibility to stakeholders.
I echo your sentiment and couldn’t agree more. We are in the process of implementing OKR’s and there is much hype surrounding the initiative. I’ve read the book and I believe there is value, but it’s more of a cheerleading “rah-rah” let’s accomplish tasks type of book. OKR’s go against the formal strategic training I’ve received in that “Goals” are the items that the organization wants to achieve. “Objectives” are the measurements of how the organization is doing regarding the achievement of the goals. “KPI’s” provide the instrumentation and data to measure how the organization or unit is doing. I am by no means an expert but quantitative data and linkage of Goals-Objectives-KPI’s just make sense. thank you again for your insightful website. It’s nice to know that I’m not alone.
Thanks for sharing your thoughts here, Jason. More of us need to talk about the limitations of the OKR framework in a measurement context, for sure.
For all of Doerr’s knowledge and experience, Measure What Matters is not a great reference text for OKRs. The examples fall well short, particularly with respect to the key results.
Good OKRs are always defined in terms of outcomes and lead measures where necessary. I wouldn’t judge OKRs from Doerr’s book, I’d check out Felipe Castro, for far better definitions and examples.
Thanks Ian for the suggestion of Felipe Castro’s work. I’ll take a look.
Well, the only thing I see you argue about is the way the objective is set. If it is set not in terms of result it is bad. OK, I agree, but this is about how to set correct Objectives. If somebody set it wrong it does not mean the system is wrong
What I debate is that the instructions and examples to guide the formation of OKRs do not make it clear how to write a good objective (because often they are written as actions milestones or projects or very vague outcomes – not measurable results), nor what a ‘key result’ really means (because often they are written as actions or milestones, not quantitative measures).
Very late to this, but you are right. MWM is full of terrible examples. OKR requires outcomes based thinking to be useful. Plenty of OKR practitioners use them to define and measure outcomes. Don’t confuse John Doerr’s examples with the reality of what happened in Google. Like any framework, OKR is only as good as the application.