Quantitative Versus Qualitative KPIs

May 21, 2013 by Stacey Barr

One of the ways that people like to classify their KPIs and performance measures is by whether they are quantitative or qualitative. Should we do it? And if we do, are we doing it right?

The distinction between quantitative and qualitative measures is often misunderstood. Really if you’re measuring anything, you’re gauging the amount to which it’s happening. And numbers are the essential building blocks of amounts. Even when you use rating scales to turn attitudes into numbers, you’re doing it to gauge an amount. So, technically, every measure is quantitative.

Qualitative measures aren’t actually measures.

In the field of statistics, we distinguish variables as qualitative (or attribute) when those variables are not gauging an amount but rather are simply putting things into buckets. The buckets are classifications like gender or market segment or geographical region or product group.

Qualitative variables aren’t performance measures. But they are used to help us analyse our measures. We can slice Customer Satisfaction Rating into product groups to explore which products to prioritise for improvement. We can dice Employee Engagement Ratio by profession and location to explore where morale might need boosting.

Quantitative measures can take two forms.

In the field of statistics, we distinguish two types of quantitative variables: continuous and discrete. Continuous variables can take any value (including decimals) over a range, and are measured in units like kilograms, hours and minutes and seconds, dollars and cents, metres.

Discrete variables are generally counts of things like complaints, accidents, new customers – anything that takes an integer value. This includes rating scales for measuring attitudes, such as satisfaction or agreement on a 10-point scale.

Performance measures can be based on either continuous or discrete variables. Measures such as Average Delivery Cycle Time or Net Profit and Non-recyclable Mass Sent to Landfill and Average Kilometres Travelled are based on continous variables. Measures such as Average Customer Satisfaction Rating and Number of Lost Time Injuries and Percentage of Projects Completed On-Time are based on discrete variables.

Both types of performance measures – continuous and discrete – are equally useful.

JOIN THE DISCUSSION:

Do you agree that discrete measures are as useful as continuous measures? What’s your reasoning? Share your suggestions on the blog.

Facebooktwittergoogle_pluslinkedinmailby feather

Speak Your Mind

Your email address will not be published. Required fields are marked *

  1. Jorgen Hansen - Denmark says:

    Sorry but I have to tell You that You are wrong about measuring quality.
    I have invented (Its operationel on 4th year) a krystal ball that can predict the outcome of quotations before they are decided – Yes and its not magic its based on behavior. However – let it be – in that process I or the tool is able to measure quality on behalf of the 80 havior caracteristics thats involved when a buyer is choising one or the other salesrep./company.

    So in regards to your article on kpi – you have some roads to walk before you can claime to know all the facts.

    Best regards – Jorgen Hansen

    PS: Be aware – I work in the real world every day and I have 6.500 quotations decided in my tool every month and 96 out of 100 predictions is correct.

    • Stacey Barr says:

      Jorgen, you have either a very dry sense of humour, or you are a magical wizard with special powers 😉

      I’m not sure how your point about predicting outcomes of quotations relates to the definition I’ve given about qualitative vs. quantitative measures. It sounds to me like you’re describing a type of discrete measurement, where you are observing behavioural characteristics and using their historic correlation to quotation outcomes as the basis for a predictive model.

      Elaborate, if you are happy to, Jorgen.

    • Joel Berman says:

      I have proven that measuring the qualitative aspect of a sales call is much more effective in changing the results of increasing revenue, profits and sales commission than relying only on numbers in the stages of the pipeline. I am working on a system that measures the quality as defined in our sales process. (playbook). Every sales person is asked to complete a stage before starting another out of the gate. The answers to questions that pertain to decision making, ability, need and criteria are just a few of probability questions that can change the direction of a sales persons time. Like in Money Ball outs cost runs. Calling on non prospects costs sales!

  2. Hisham Ismail says:

    Just wanted a clarification concerning the “Qualitative” measures. Does this mean that we only depend on the quantitative whether its discrete or continuous?

    Also i would like to know why wouldn’t we include a KPI like the (uCstomer satisfaction) under the qualitative measure since we are here talking about (The quality of service provided)?

    • Stacey Barr says:

      ‘Qualitative’ and ‘quality’ have two different meanings.

      In statistics, ‘qualitative’ is as I have described in the article: non-numeric data.

      In business, the word ‘quality’ is used a lot to mean effectiveness or how well something is done or achieved. You can measure how well something is done or achieved using either continuous measures (such as cost of rework or cycle time) or discrete measures (like average customer satisfaction rating).

      You can seen how terminology and variation in our definitions of words can get us all confused pretty quickly! In fact, I wonder if the comment that Jorgen made on this post is indicative of the same terminology mix-up?

  3. Hisham Ismail says:

    I meant (Customer satisfaction)*

  4. Terence says:

    Discrete and continuous varaibles share the same characteristics when they are collected in large number sets. For the purpose of operational use of KPI’s, your differentiation is not relevant. The attempt is always to measure the qualtity of work or product and the quality of work or product ( and to draw some conclusions from the comparison of the KPI’s.

    • Stacey Barr says:

      Terence, thanks for your comment.

      The only reason I believe my differentiation of ‘discrete’ from ‘continuous’ is relevant is the confusion that so many people have about the difference between measuring quality and using qualitative data. They are very different! You can see by the other comments on this post that this confusion is real.

  5. Rich Torr says:

    How would you classify ordinals (high, medium, low) and rankings in your definition Stacey? Do you find that people associate the use of ‘Qualitative’ variables with measuring ‘Intangible’ objects?

    • Stacey Barr says:

      Ordinal data and rankings are usually considered discrete measures, Rich. And yes, I think people are confusing ‘qualitative’ measures/variables with intangibles, too. Again, our terminology holds us back from understanding!

  6. Vikas Khandekar says:

    First of all, without getting into too much of semantics, I agree with you that every measure is quantitative. If one cannot quantify something, one will not be able to measure it, so by corollary, everything that is measurable is quantity driven – quantitative. The trouble comes when we talk about “qualitative” measures, for example about the taste of food or a beverage. In a survey question “Did you like the taste of the sample wine?”, the answers are going to be very varied if the choice given to the respondent is unlimited. Then the analysis of the feedback becomes next to impossible. The work around to the situation, is to take subjective feedback in discrete buckets of “Excellent”, “Good”, “Neutral – nothing great”, “Not so good”, “I hated it”… and then perform the analysis which will strictly be quantitative. Hope the above example helps resolving some of the terminology related confusions and also to further corroborate Stacey’s point of view about quantitative and qualitative measures.

  7. Sunny Khoo says:

    Hi Stacey, love your article. And fully agreed with your reasoning. I have another point of view – using the concept of effectiveness vs. efficiency KPIs. “Effectiveness” is liken to outcome driven measures (usually is output related) while “efficiency” is linked to means to an end, i.e. efforts. In that way efficiency KPIs can be comparable to qualitative as to effectiveness to quantitative.

    • Stacey Barr says:

      Sunny (pretty name!), I agree. This idea of effectiveness versus efficiency is something lots of people talk about. They are two domains of performance, and like you describe. Effectiveness is about how well a process produces its intended outcomes/results. Efficiency is about how well a process uses resources to produce its intended outcomes/results.

      I would not, however, agree that they parallel qualitative versus quantitative, as I’ve described them above. Again, I think there is a confusion here that qualitative is the same as intangible. There really is no true qualitative measure. Qualitative means no numbers, therefore no quantification. Rather, as I describe in the article, you use qualitative variables to classify or segment or categorise the quantitative measures.

      So you would have both efficiency and effectiveness KPIs as being quantitative. Efficiency, for example, can be measured by ‘total production cost per product sold’ or ‘total hours spent per help desk problem solved’. Effectiveness can be measured, for example, by ‘average customer satisfaction with product quality’ or ‘percentage of help desk problems solved to customer requirements’.

  8. Dr,Helmy Ismail says:

    After reviewing the measurements, we need the two types, the quantitative and qualitative.
    For example, when setting the KPIs. for the key jobs, we may set both measurements to asses the performance of some jobs specially for sales and production staffs. Also, for some jobs which related to accidents, incidents, we need the qualitative measures.
    This means that the two measures give more flexibility in assessment processes.
    This is my viewpoints,
    Dr. Helmy Ismail

    • Stacey Barr says:

      Interesting. I think like a few others in this discussion Dr Helmy, when you refer to a qualitative measure, you mean a measure of a less tangible result, and not the same meaning as I describe in the article. There are no truly qualitative measures: but we can make quantitative measures for qualitative attributes.

  9. Ben Slome says:

    @Joel Berman,

    Would love to hear more about your prediction model as I believe I could have use for it in my business, would you mind getting in contact with me?

  10. Menes Rafael Garza says:

    Although scenarios, DOE’s and many others requires as a input data from baseline, many companies manage theirs improvement truly on P&L.

    Some experts rely on KPI’s clasificación as per it’s executions results dimensions, as Efeciency and Efectivity which as a criteria sound reasonably since management on dimensions creates this levels of execution and KPIS’s measure primarily the level of exceution.

    Menes Rafael Garza

    • Jeje says:

      I think Stacey is right..Suppose you want to measure the “willingness” of managers in executing a particular strategy. Although the term “willingness” sounds “qualitative”, you will have to rate it e.g in percentages or in any ordinal metric before you develop your analysis on managers’ willingness.

      Jeje

Connect with Stacey


Haven’t found what you’re looking for? Want more information? Fill out the form below and I’ll get in touch with you as soon as possible.



*We respect your email privacy.
PO Box 422
Samford, Qld, 4520
Australia
Stacey Barr Pty Ltd
ACN: 129953635
Director: Stacey Barr
Simple Share Buttons