Skip to content

How Human Insight Helps Make Sense of Big Data

Big data is a $122 billion industry. Yet 73% of big data projects are unprofitable, according to a survey by Capgemini and Informatica. 

So why isn't more data helping us make better decisions? What's missing?

Those are the questions posed by technology ethnographer Tricia Wang in her TED talk, 'The human insights missing from big data'. In her presentation, Wang tells the story of how phone company Nokia -- once the world's largest mobile phone maker -- collapsed when it failed to listen to what customers needed and anticipate approaching trends. 

It's a cautionary tale for companies who over-rely on big data and ignore its limitations

Let's back up a moment: Big data is very useful for quantifying specific environments, like energy usage or delivery logistics. The problem, says Wang, comes when we try to quantify dynamic systems -- especially ones that contain a human element. 

To understand why, consider this situation: A patient walks into an emergency room with nausea, dizziness, and chest pain. When the tests come back normal, doctors chalk it up to stress and send the patient home. Days later, the patient is back -- this time in the middle of a full-blown heart attack.

You wouldn't expect a doctor to dismiss the obvious signs of a heart attack because the tests looked fine. So why would you dismiss human insights simply because they can't be measured? 

Yet when it comes to big data, we're often inclined to take the phrase "You can't manage what you can't measure" too literally.

That's especially relevant to safety leaders, because there's always a human element. No matter how much quantitative data we collect on injury rates, circumstances, and days away from work, it will never be able to tell the whole story. 

The solution, says Wang, is to focus on "thick data" -- precious, unquantifiable insights from actual people -- and to stop throwing out data simply because it can't be put into numbers. 

So what does work? Integrating big data with thick data to form a complete picture. 

Says Wang, "Big data is able to offer insights at scale and leverage the best of machine intelligence, whereas thick data can help us rescue the context loss that comes from making big data usable, and leverage the best of human intelligence." 

Naturally, gathering thick data -- stories, emotions, and interactions that can't be quantified -- poses a unique challenge.

How do we do that in the real world? After all, not every company can afford to hire a data ethnographer like Wang. However, companies can strategically collect employee observations -- both qualitative and quantitative -- to form a better picture of their risks and potential solutions.

Today, mobile EHS apps are making it easier than ever for companies to collect, analyze, and act on employee observations:


  • Workers can capture observations, images, and video using any smartphone or tablet
  • Supervisors can customize forms to collect any data they want, asking closed questions like "When and where did this happen?" as well as open-ended questions like "Why did this happen?" and "What can we do about it?".
  • Leaders can analyze the data in dynamic dashboards, drilling down on each event with an added layer of context provided by qualitative observations

Bottom line

Human insight is a powerful tool to make sense of big data. By combining this "thick data" with analytics, companies can connect the dots and ultimately make better decisions. 

New call-to-action