When it comes to cause analysis for performance measures, many people put the cart before the horse. They rush too quickly to analyse their data without first establishing a clear purpose.

The purpose of any kind of data analysis is to answer one or more questions.

My framework for getting from questions to answers when you’re doing cause analysis for performance measures has nine steps:

STEP 1: Phrase your question

Write down the question you want to answer and make it as specific as possible. When you’re doing a cause analysis for a particular performance measure, it can help to first brainstorm the factors that might impact that measure and then form one or more questions about each factor. Focus on questions that are useful to answer, not just interesting!

For example, “Do staff with longer tenure cause less rework?” or “Is rework different for different types of tasks?”

STEP 2: What form of answer?

What will you compare with what to answer your question? What are the possible answers to your question and how do you imagine the way this answer would look on paper?

For example, to answer the question about whether staff with longer tenure cause less rework, you want to compare tenure of staff against rework of staff. To answer the question about whether rework is different for different task types, you want to compare the amount of rework for each task type.

STEP 3: What analysis type?

What data analysis process will you need to apply to produce the answer in the form you need it? Do you need a summary, exploratory, explanatory or predictive analysis technique?

For example, an explanatory analysis technique to compare tenure of staff against rework of staff is a regression analysis. An exploratory analysis type to compare the amount of rework for each task type is a Pareto chart or simple bar chart.

STEP 4: What data?

What types of data does your analysis need as an input? List each piece of data you need, its format (eg time on a 12 hour clock versus a 24 hour clock), its value range (eg a scale of 1 to 5), its units (eg centimetres or metres) and its precision (eg dates as month/year versus day/month/year).

For example, the data for a correlation analysis of tenure of staff against rework of staff would be a list of staff members and for each of them a value for their tenure and a value for the rework they generate. The data for a simple bar chart of rework for each task type is a list of the task types and for each of them a value for the total amount of rework generated.

STEP 5: Get the data

Do you have this data already available? Or do you need to collect it? Plan your steps in accessing the data that is available, making sure you understand its degree of integrity. Plan your steps in developing a data collection system for data that you don’t already have.

For example, the data for a correlation analysis of tenure of staff against rework of staff could come from the payroll system and also check sheets where people have recorded the amount of rework generated. The data for a simple bar chart of rework for each task type could come from the check sheets where people have recorded the task type along with the rework generated.

STEP 6: Get to know the data

Bring all of your data together in a place (such as a spreadsheet or database) in preparation for using it. Look over some of the raw data to visually check if it looks okay. Play with some summary statistics like minimum and maximum values, totals and means to check for outliers or potentially erroneous data values.

For example, looking at the minimum and maximum values for staff tenure might reveal that one staff member has 123 years of service, which quite obviously, can’t be right.

STEP 7: Perform the analysis

Apply the analysis you chose, and check over the results to make sure they look plausible. If you find anything funny, go back and check your data integrity. Get help to perform the analysis if you don’t have the confidence or the skills.

For example, create a scatter plot of tenure of staff against rework of staff and get your analysis software (even Excel will do it) to put a regression line or equation on the chart to quantify the relationship. Summarise the data for rework by task type by calculating total rework for each task type and creating a bar chart.

STEP 8: Interpret the results

Translate the output of the analyses into everyday language. Describe clearly the patterns or relationships that the analysis uncovered in your data. Make sure you include information about how reliable those patterns or relationships are, based on your data’s integrity and the kind of analysis you used.

For example, determine if the relationship between staff tenure and staff rework is a strong one or weak one by looking at both the scatter plot’s variability and at the measure of statistical significance, the R2 value. In the bar chart of rework by task type, are there any specific tasks that have significantly more rework than others, or are they all pretty much the same?

STEP 9: Answer your question

Draw conclusions from the interpretations in the context of the question or questions that triggered the analysis. Notice if any new questions surface. If so, go back to step 1 to phrase your new questions.

For example, if you see a strong and statistically significant relationship between staff tenure and staff rework, you’ve answered your original question “Do staff with longer tenure cause less rework?” with a “yes”. A new question might be “Why do staff with shorter tenure cause more rework?” and you’d start the process over again. If you don’t see any particular task type exhibiting more rework than any other, and they all look roughly the same, then you’d answer your original question of “Is rework different for different types of tasks?” with “no”. And you might then phrase some new questions to explore other possible causes of rework, such as “Does time of day impact rework?”

TAKE ACTION:

Do you leave out any of these essential steps to data analysis? If so, create a checklist from these steps to guide you systematically next time you need to use data to answer a business question.

1. Trisha says:

   July 30, 2013 at 12:05 pm

   Hi again Stacey,

   Would love some more info on the analysis techniques section. Any tips on further reading?
   Trisha

   Reply
   • Stacey Barr says:

     July 31, 2013 at 3:18 pm

     Trisha, there are loads of books on analysis, but keeping it simple is the best way to go. Here are a few additional posts about resources that might help, particularly in the context of performance measurement:

     https://staceybarr.com/measure-up/book-review-show-me-the-numbers-by-stephen-few-2/

     https://staceybarr.com/measure-up/are-you-reacting-to-trends-that-arent-really-there/

     https://staceybarr.com/measure-up/the-business-questions-your-performance-measures-should-answer/

     https://staceybarr.com/measure-up/book-review-understanding-variation-by-donald-wheeler/

     Reply
2. Bill Ravensberg says:

   July 31, 2013 at 12:42 am

   What a great breakdown! I really like the graphic too!

   It’s often so easy to try to jump to the answer from the question but that usually means we don’t always understand the question. Oh oh!

   This process will serve well to ensure that all aspects are considered in answering the question and should lead to identifying any possible outside influences too when you “get to know the data”.

   Thanks Stacey for putting something that most people consider as being simple into a way of presenting the critical thinking that really needs to occur!

   Reply
   • Stacey Barr says:

     July 31, 2013 at 3:19 pm

     Thanks for your kind feedback, Bill! I particularly agree with your comment about analysis needing critical thinking… not just familiarity with how to use Excel!

     Reply
3. Prahlad says:

   July 31, 2013 at 3:20 pm

   Stacey,

   I would like to congratulate you for writing blogs on KPIs which are very helpful for me in enhancing my knowledge and execution at work. I wanted to make a point and get your thoughts on it.

   You are suggesting us to watch the KPIs on a control chart and watch the movement of the KPI signals over LSL & USL lines (+/- 3 sigma limits).

   I feel that this is a good approach for detecting special cause variations. But I would spend very little time on these to understand the cause of these assignable variations. Also once I know the cause of these variations – I will take them out of my measurement. My focus would be on reducing the mean or random cause variations of a process by understanding the causes……..To me best measure for all the processes is SIGMA score or Z score and I feel this should be watched instead of just focusing on special cause variations in a process which is noise in the measurement

   What do you think?

   Regards
   Prahlad

   Reply
   • Stacey Barr says:

     July 31, 2013 at 4:24 pm

     Prahlad, thanks for the comment!

     I love using XmR charts (the best type of control chart for management performance measures). I would suggest though, that points falling outside the upper and lower Natural Process Limits is just one type of signal to look out for and respond to.

     The Natural Process Limits, along with the Central Line, are there in XmR charts to stop us from reacting to noise. Donald Wheeler, in his book Understanding Variation, details this. I don’t think many decision makers have these basics right yet, and I wouldn’t be keen to offer up z scores.

     Here is a link to my review of Donald Wheeler’s book:
     https://staceybarr.com/measure-up/book-review-understanding-variation-by-donald-wheeler/

     Perhaps I’ll write a blog post on the signals that XmR charts can show us, but in the meantime, here is what I have written: https://staceybarr.com/?s=xmr

     Reply
4. Prahlad Bhugra says:

   July 31, 2013 at 9:58 pm

   Thanks Stacey, I look forward to reading your blog on ‘the signals that xmR charts shows’. Also, I will purchase a book by Donald and read- looks to be an insightful book.

   I was wondering if some one is wanting to compare the key processes in his/her organization on a uniform scale (which processes is better than other) and/or wants to incorporate the specification & target in his analysis – how can he do with absolute value of KPI plotted on an xmr chart.

   regards
   Prahlad

   Reply
5. Analysis paralysis, why it kills productivity and how to overcome it – Clockify Blog says:

   April 1, 2021 at 1:18 am

   […] Barr, a performance measure specialist outlined a great guide on how to avoid analysis paralysis. When it comes to structuring your research process so you don’t trigger the vicious cycle of […]

   Reply