Don’t Blindly Trust the Data

The business world is obsessed with data, and for good reason. New technologies allow us to collect and analyze huge swaths of data. And the smartest companies out there use that data to find actionable insights, things they can use to help improve efficiency and grow the company.

But in order to make sense out of the data, human beings need to get involved – to dissect, analyze, and draw conclusions. And that’s where things can go wrong.

Don’t blindly trust the data. We need to learn how and when to question it in order to get the most from it.

Sometimes the data will suggest something that common sense will tell you is wrong. Sometimes one metric may look good while competing metrics do not. Sometimes a problem you thought you had turns out to not be a problem at all.

What can go wrong?

  1. The sample size may be too small. You can’t draw real conclusions unless you have a statistically significant result. When the sample size is too small, one data point can have an overwhelming effect one way or the other.
  2. The analyst incorporated his or her own bias. It’s easy to mold any dataset to fit some preconceived belief. The data should guide the theories and not the other way around.
  3. You may be asking the wrong questions. When you start with a question, you look at the data that you think answers that question. And you might ignore a more important data set that you otherwise would have looked at with a more appropriate question.
  4. The visualizations used don’t tell the full story. There are a great many common mistakes people make when trying to tell a story with visualizations. Charts and graphs can be distorted to over- or under-sell a specific story.
  5. Confusing correlation and causation. Just because the data shows you that two things are true about a certain population, doesn’t mean that one thing causes the other.

These are just a few common things that can go wrong when companies try to draw actionable insights from the data available to them. By learning more about how to work with data, managers and decision makers can prepare to question the data that they’re getting, rather than blindly following the recommendations by their data scientists and analysts.

The key is to know what can go wrong, that way you can avoid it.