Government outcomes are particularly challenging to measure meaningfully, because they are intangible, long-term, and seldom directly impacted by a single department’s or agency’s actions.

The current common practices for choosing measures of performance are too limited to meet these challenges of creating good outcome measures for government departments and agencies.

CHALLENGE 1: Government outcomes are intangible.

Most government agencies exist to influence social stability, justice or welfare; not to produce widgets. Inherently, these social outcomes are softer and fluffier than the outcomes of non-government organisations. But this isn’t the reason that outcomes are hard to measure. If an outcome matters, we should be able to describe it clearly enough to observe, in some way, if it’s happening or not. But how most government agencies describe their outcomes is not with clear language that describes anything observable:

“The advancement of Australia’s international strategic, security and economic interests including through bilateral, regional and multilateral engagement on Australia Government foreign trade and international development policy priorities.” – Foreign Affairs and Trade

The common practice that limits our ability to meaningfully measure intangible outcomes is using vague ‘weasel words’ to articulate those outcomes. If we accepted the premise that some outcomes are too broad and intangible to measure directly, then we accept the premise that there will be no evidence of such outcomes if and when they happen. If there is no evidence of an outcome when it happens, no-one will notice if it’s happening or not. So why bother with all the effort to try and create that outcome in the first place?

To succeed at finding meaningful measures of intangible outcomes, we need to use plain language to articulate them. And we can’t start with questions like “How could we measure this?” Rather, we need a measure design process that starts with questions like “How would we recognise if this was happening or not?”

CHALLENGE 2: Government outcomes are long-term.

If it’s true that government agencies exist to influence social stability, justice and welfare, then it can’t be ignored that such things change over longer timeframes. Most government outcomes can’t be measured as frequently as sales quotas or revenue growth or tonnes of coal exported:

“Contribute to the improvement of the extent, condition and connectivity of Australia’s unique biodiversity and natural resources, consistent with national and international obligations, through protection of habitats and mitigation of threats to threatened species and ecological communities.” – Department of the Environment and Energy

The common practice that limits our ability to meaningfully measure long-term outcomes is limiting our measures to our corporate planning horizons. And this usually means that we measure only the strategic initiatives within those planning horizons. The above outcome isn’t measured directly, and only its Delivery Strategies have measures (and most are activity measures or milestones):

“Number of projects/contracts.

Project designs and contracts reflect the objectives of the Reef 2050 Plan and Reef Trust.

Project outcomes are fed into modelling and monitoring programs to inform reporting on Reef health.”

To succeed at finding meaningful measures of long-term outcomes, we can’t default to monitoring strategies and initiatives. If we do, we’ll struggle to ever learn which types of activity have the best impact on the outcome. In contrast, when we measure both the outcome and the results of our strategies and initiatives, we can test which work best, and then develop useful lead indicators to monitor more regularly to project changes in the outcome measure.

CHALLENGE 3: Government outcomes are systemically complex.

The role of government is more to influence than to directly control social outcomes. Government organisations often rely on collaboration with other public, private and nonprofit sector organisations for an outcome to be achieved. Government departments and agencies can struggle to isolate their contribution toward outcomes they can only partially influence, or ‘facilitate’:

“Facilitate jobs growth through policies and programs that promote fair, productive and safe workplaces, and facilitate the growth of small business.” – Department of Jobs and Small Business

The common practice that limits our ability to measure systemically complex outcomes is defining accountability as ‘hitting targets’. We know that the actions of a single department or agency cannot completely control if or how well a systemically complex outcome will happen. No one will accept being judged by a target they don’t have control of achieving. So, it’s no wonder that the above outcome is measured by such an operational measure as this:

“Percentage of claims processed within 16 weeks of receipt of an effective claim.”

To succeed at finding meaningful measures of systemically complex outcomes, we can’t waste time trying to isolate our own department’s or agency’s impact on those outcomes. Rather, we need to build and test a causal map of how we influence that outcome, and measure both the outcome and its drivers (causal factors) that we do control.

These challenges are the clues to how to measure government outcomes.

This white paper on How to Measure Government Outcomes goes into a lot more detail on dealing with these challenges, and offers a long list of techniques and resources to make the job easier.

Government outcomes can be meaningfully measured by working through the challenges of how intangible, long-term, and systemically complex those outcomes are.
[tweet this]

WATCH OR SHARE THE VIDEO: