How Accurate and Reliable is Your Data?

Six Sigma is a process improvement methodology that uses facts and data to make decisions, not hunches and opinions.  It is obviously important that we use facts and data that are accurate and reliable.  The more important the decision and the more serious the consequences, the more care we need to take when collecting data.

The Supreme Court of the United States is continually faced with making decisions that can affect millions of people and cost billions of dollars.  Some of the issues that are brought before the court are strictly legal, but many turn on more complex questions of policy.

ProPublica released a study in October, 2017 that was discussed in the NY Times on October 18, 2017 in an Op-Ed article by John Pfaff entitled The Supreme Court Justices Need Fact Checkers.  The study suggests that the justices on our Supreme Court struggle with the task of using data effectively to make decisions.  ProPublica found that the court used faulty research or introduced their own errors in nearly one third of the 24 cases reviewed that relied on such facts.

Here is one example.  In 2013, the decision rendered by the court on Shelby County v. Holder invalidated a portion of the Voting Rights Act of 1965, making it one of the most consequential decisions in recent years.  The arguments presented by Justice John G. Roberts, Jr. suggested that the South had made great strides in registering voters, rendering the protections of the act unnecessary.   He based his decisions in part on a Senate Judiciary Committee analysis that misinterpreted how the Census Bureau reports data on race and ethnicity.  Justice Roberts wrongly asserted that registration gaps between minorities and whites had shrunk significantly, an error that no one caught.

The results of the ProPublica study are troubling but not surprising.  None of the justices are trained in statistics, and the clerks who assist them are virtually all law school graduates who rarely have any formal training in the use of data and statistics.  Facts are central to the decisions that the justices make in a number of their policy related cases, but its members lack expertise.

I see the exact same situation in business and industry as I mentor and coach Six Sigma students at all levels up to and including the Master Black Belt.   People who lack any formal training in how to properly collect and analyze data make decisions that are based either on faulty data and analysis, and/or based on their personal biases and opinions rather than on facts.

We must always be vigilant to make sure that we are using data that has been properly collected and analyzed when making decisions about how to improve the performance of our organizations.

Your comments or questions about this article are welcome, as are suggestions for future articles.  Feel free to contact me by email at

About the author:  Mr. Roger C. Ellis is an industrial engineer by training and profession.  He is a Six Sigma Master Black Belt with over 50 years of business experience in a wide range of fields.  Mr. Ellis develops and instructs Six Sigma professional certification courses for Key Performance LLC.   For a more detailed biography, please refer to