# Statistical Software – a Trap for the Unwary

In our Six Sigma courses we make extensive use of Minitab statistical software.  Minitab is an extremely comprehensive and powerful software package.  At the same time author Milton Rosenau provided this caution when he said “(Software)…is also a trap for the unwary.”

When evaluating the output of a piece of software such as Minitab, we must ask ourselves several questions.  Does the output make sense?  Did we generate the correct graph or use the correct analysis routine for the data?

Here is an example from my recent teaching experience.  One of my Black Belt students generated a control chart to establish baseline performance for the process that he plans to improve.  He is tracking defectives (one or more problems with a form as submitted makes it defective) and the area of opportunity is relatively constant, so a p chart seemed to be appropriate.  The goal of his project is to reduce the rate of defectives.  Here is the control chart that he initially submitted:

The chart was intended to be generated for the proportion of defectives.  When I examined the underlying data, I confirmed that the chart in fact is for the proportion that were correct the first time (which is .6565 or 65%) instead of the proportion that were defective the first time.

I was also concerned about the control limits on the chart, which at first glance seemed far wider than what one would expect.  I reran the chart using data for the proportion defective and got the following result:

The chart now correctly shows the proportion of defectives (an average of .3435 or 34%), but the control limits are still excessively wide.  I ran the p chart diagnostic routine in Minitab and got the following in return:

Note that the return from Minitab states the following:  “Using a p chart may result in a chart with control limits that are too wide for the data.  Consider using a Laney P’ chart instead”.  I ran the Laney P’ chart for the proportion defectives, and here is the resulting chart:

This chart shows a very different story than the chart that was first submitted by the student.  The initial chart did not use the underlying data correctly, nor did it use the type of chart that was most appropriate for the data.  The result was a chart of the wrong variable with control limits that did not make sense.