How to Audit Performance Measurement in Government

by Stacey Barr

Despite the existence of government performance measurement audit processes, performance measurement in government isn’t improving. This is why…

Charts and stethoscope representing measuring a government agency. Credit: https://www.istockphoto.com/portfolio/juststock

For a long time now, governments have been trying to get a stronger alignment between agency outcomes and the budgets they are allocated to produce those outcomes. And so the purpose of auditing performance measurement in these government agencies is to create more accountability for this, and provide feedback on how to improve it.

There are many governments around the world that provide performance frameworks to guide how their agencies should measure and manage performance. Searching for the details of the overarching model and criteria of such frameworks, however, is very difficult. Each country has a confusing labyrinth of webpages and pdfs that relate to performance auditing. But at least some examples of government performance frameworks include:

Despite the existence of these performance frameworks, and the audits that can follow, government agencies are still not demonstrating excellence in performance measurement, nor the achievement of their outcomes. How can this be?

Government audit frameworks have limitations.

Audit offices are experts in auditing, not performance measurement. Couple that with the requirement governments so often have to be impartial to any proprietary method. The result is that their performance frameworks list
criteria or requirements that can be vague, incomplete, prescriptive and somewhat random. For example:

  • “include measures of an entity’s outputs, efficiency and effectiveness” – one of six requirements for performance indicators from the PGPA
  • “a process is in place to establish indicators and targets” – one of five audit areas used in Canada’s Audit of Department Performance Measurement
  • “balanced, giving a picture of what the organisation is doing, covering all significant areas of work” – representing the ‘B’ in the UK government’s FABRIC model of evaluative criteria

On top of this, each governments’ frameworks emphasise a different set of criteria or requirements. Which country has it right, and how wrong is every other country?

Shouldn’t there be one complete framework of performance management that guides government agencies to measure how well they provide the outcomes they exist, and are funded by taxpayers, to deliver?

The criteria of a performance framework needs a framework.

It’s not enough to give a set of unholistic guidelines as a performance framework. To improve government agency performance, we need a fundamental shift in what performance and its measurement really means. And audit frameworks and processes have the opportunity to facilitate this bigger shift.

Business excellence frameworks around the world – like ABEF, the Baldrige Award and EFQM – have done the thinking about how to build evaluation frameworks that stand the test of time, that really do guide best practice. And they do this without being vague or prescriptive.

One part of their evaluation framework is the concept of ADRI (borrowed from the ABEF). ADRI stands for Approach, Deployment, Results and Improvement. To evaluate an organisation against the business excellence framework, we look for evidence of these four parts, against each category of the framework.

Business excellence frameworks include categories that directly relate to performance measurement, like these in the ABEF:

  • Category 5: Information and Knowledge
  • Category 7: Results and Sustainable Performance

Therefore, in evaluating government agencies against a performance framework, we can learn from these business excellence frameworks to get more rigour and depth.

For the purpose of discussion, I present to you an example of how government performance audit frameworks could be designed.

Let me stress: I am providing examples only, and this is most definitely not a complete set of criteria for auditing performance measurement!

Approach – is there a robust performance measurement method?

ADRI’s Approach criteria would evaluate if the agency has a deliberate and robust method for its performance measurement process.

For anything to be auditable, it has to have a methodology. So, for performance measurement to be auditable, there needs to be a performance measurement methodology. What’s the point in auditing random KPI behaviour that changes from organisation to organisation?

A performance measurement audit framework would then have specific criteria relating to the approach adopted for performance measurement. For example:

  • There is an approach for the selection, implementation and use of performance measures throughout the agency.
  • The approach ensures the agency can create the performance measures that evidence its outcomes and the drivers of those outcomes.
  • The approach guides how performance measure calculation, data requirements, interpretation and reporting requirements are decided and documented.
  • The approach makes it clear how performance measures are reported and interpreted to identify performance gaps that need to be closed.

Deployment – is the performance measurement method consistently implemented?

ADRI’s Deployment criteria would evaluate how consistently the agency is implementing its performance measurement approach.

In particular, the implementation of the approach would consider its consistency over time, the breadth of its use across all organisational functions and processes, and the depth of its use from the executive team to front-line teams.

A performance measurement audit framework would also have specific criteria relating to the deployment of the performance measurement approach. For example:

  • The right people are involved directly in developing the performance measures they will use.
  • All performance measures developed are aligned to the agency’s strategic goals and purpose (mission and vision).
  • The approach is implemented as part of the routine work of strategy development, performance reporting, decision making, and performance improvement.

Results – how well is performance measurement working?

ADRI’s Results criteria would evaluate how well the agency’s performance measures are helping to improve agency performance.

At the risk of making your brain bend back onto itself, here we are taking about measures of performance measurement. What is the evidence of performance measurement working well? If we agree that the role of measurement in government is to help reach the outcomes promised with the budget provided, then we have a starting point.

A performance measurement audit framework also needs specific criteria relating to the results of the performance measurement approach. For example:

  • Performance measures are trusted and used by their owners.
  • Performance measures show progress toward targets, or targets are met.
  • Unintended consequences resulting from use of the performance measures are managed or mitigated.
  • The return gained from the performance measurement approach (e.g. amount of performance improvement) is greater than the cost of performance
    measurement (e.g. data, reporting).

Improvement – how does performance measurement get improved?

ADRI’s Improvement criteria would evaluate how well the agency has evolved and improved its approach to performance measurement, based on reflection on how well it works (ADRI’s Results).

Do they ever reflect on how well it’s working? What is the agency learning and improving about their performance measurement approach? If they see it’s not working well enough, to they find the causes and improve their approach? What’s the evidence that changes made to its performance measurement approach actually changed the results of how well it works?

A performance measurement audit framework finally needs specific criteria relating to the improvement of the performance measurement approach. For example:

  • The results of the performance measurement approach are routinely reviewed to identify where it works well and where it needs improvement.
  • The performance measurement approach is only changed based on evidence of where it is not working well enough.

***

Do you also believe we need to take a serious look at how our government organisations are being encouraged, by way of audit, to measure their outcomes? What’s happening now isn’t working. I wrote this article for discussion. I don’t have the answers, and I am keen for you to share your thoughts.

Shouldn’t there be one complete performance framework to audit how well any government agency on the planet provides the outcomes they exist, and are funded by taxpayers, to deliver? [tweet this]

FacebooktwitterlinkedinmailFacebooktwitterlinkedinmail

Leave a Reply to Kirsten Makins Cancel reply

Your email address will not be published. Required fields are marked *

  1. Kirsten Makins says:

    This is a really interesting article. It makes a great deal of sense to measure how well performance measurement is being done in an organisation (I work for NatureScot, a non-departmental public body in Scotland). It would help drive development of the process and how it is adopted and managed. (Arguably, it’s not just governments that need to do this, but I assume that was a given in the thinking.)
    It has allowed me to shape something I’ve been trying to grasp for a while (bear with me, I’ll get round this huge circle shortly) – a maturity assessment is a common way of benchmarking your organisation against the process (benchmarking against others I find less helpful, because their priorities are different, their culture, their management style, etc). In the world of PRINCE2 and MoR (management of risk) there are maturity assessments that give a description to each rating (1-5). The equivalent that comes with PuMP has a 1-10 rating, but from only two descriptions (poor/excellent), making it harder (for me, anyway) to see how I make incremental steps towards excellent. The addition of descriptions in the prince2/MoR gives a clearer picture of how one might move from rating to rating, across a number of categories. I can see that measures allowing one to track progress from one rating to another would be a sensible thing to do to increase performance measure maturity. I would love to have something along these lines to help us mature in our approach. Might it need to be a twin approach – designing measures for the process alongside a more detailed description of maturity, from beginner to expert?

    • Stacey Barr says:

      Kirsten, this is tremendous. I wanted to upgrade the PuMP Diagnostic (the tool you refer to) to make it a way to measure baseline and progress changes in the maturity of performance measurement, but hadn’t thought about giving clearer descriptions of each level of rating. Thanks so much for the idea, and for moving me a step closer to building such a tool!

Upcoming KPI Training


Our next PuMP Performance Measure Blueprint Workshops include:

>> North America, Online Interactive, 7-11 Feb 2022

>> Australia/NZ/Asia Pacific, Online Interactive, 23-24, 28 Feb, 1-2 Mar 2022

>> Africa, Online Interactive, 14-18 Mar 2022

>> Australia/NZ/Asia Pacific, Online Interactive, 29-31 Mar 2022

>> Africa, Online Interactive, 20-24 Jun 2022

>> Africa, Online Interactive, 26-30 Sep 2022

>> Africa, Online Interactive, 28 Nov - 2 Dec 2022

Register for the next PuMP Blueprint Workshop near you

Stacey's Books


Prove It! How to Create a High-Performance Culture and Measurable Success, book by Stacey Barr

Order Prove It! here.

Practical Performance Measurement: Using the PuMP Blueprint For Fast, Easy, And Engaging KPIs, book by Stacey Barr

Order Practical Performance Measurement here.

Reprinting Articles


You are welcome to use articles from the Measure Up blog, with these requirements

Connect with Stacey


Haven’t found what you’re looking for? Want more information? Fill out the form below and I’ll get in touch with you as soon as possible.



    *We respect your email privacy.
    Level 54, 111 Eagle Street
    Brisbane, Qld, 4000
    Australia
    Stacey Barr Pty Ltd
    ACN: 129953635
    Director: Stacey Barr