Surveys are one of the common methods we use in business to gather data. We have customer surveys, employee surveys, corporate image surveys, and surveys just about anytime we want to collect data from people. And often you’ll see these surveys listed in the KPI column in the business plan. BIG MISTAKE! Here’s why:

Surveys are really data collection processes and not performance measures at all. And the same goes for other data sources you might see listed in the business plan’s KPI column:

• ‘Customer Survey’
• ‘Occupational Health and Safety incident reports’
• ‘Budget’
• ‘SAP system’

How exactly does a data source provide evidence that a goal has been achieved? Of course the data source provides data about the goal, but which data specifically? And how exactly should those data be analysed to answer the question of whether the goal is being achieved?

The above so-called measures are means of collecting data, but they do not provide the answers to our questions about performance.

Performance measures can certainly be BASED on the data that these processes collect. But the measures need to be clearly designed and defined before the right data can be collected.

When I worked in the Queensland Government Statistician’s Office as a survey statistician, I faced over and over the problem of data being collected without a clear idea of how they would be used. Our clients would first list all of the data they wanted to collect in a survey without making it clear what they would use those data for or what business questions they needed to answer.

Seeing data collection processes listed as performance measures is a symptom of not really knowing what the organisation’s actual performance measures are. Identifying a data collection process suggests that either we have only an inkling of what kind of data might relate to our goals or that we lack discipline in being clear and specific!

Measures should be designed before data is selected, for the same reasons that a house should be designed before building materials are selected. If we start with the data, we risk ending up with the wrong measures and missing out on getting the data we really need for the right measures. Or we risk ending up arguing about exactly how to use the data to decide whether our goals are being achieved or not. Neither of these outcomes gives us a true measure of performance.

TAKE ACTION:

Do you have any data collection processes that assemble lots of the wrong kinds of data?

Try to write down the business questions that you most need your data collection processes to answer, and then identify the form that the answers to these questions should take. From there, you can identify the analysis that can produce these answers and the data that this analysis would require. Once you’ve done this, you will be much closer to knowing what your performance measures really are (as well as the best data to collect to measure performance).