The Data Overload

It has been said that oil was the commodity of the 20th century, but data will be the commodity of the 21st century. In the past 30 years, computer processing power has grown exponentially, and with it, the capacity and ability of our technology to gather data has skyrocketed. Statisticians and data scientists have been grappling with the very real problem in recent years of how to analyze and make meaning of the copious amounts of data that they now have access to, but I assert that we as a society are also struggling with the data overload in our day-to-day lives.

For example, it would appear that police interactions and mishaps with increasingly angry communities are on the rise. It would also appear that gun violence, divorce rates, and even global temperatures are increasing at alarming rates. In all of these cases, I believe that the onslaught of new data that we have, that weren’t available in accessible forms decades ago, are influencing or even misleading these trends.

Take the police interactions today. It is entirely possible that body cams, cell phone footage, and 24-hour news coverage have made these types of situations appear many times more common than in days past, even to crisis levels, when perhaps the prevalence is really not much different than it has ever been in the past few decades. That is not to say there is not a real problem, just that the problem has be extant for many decades and isn’t a new phenomenon.

Or gun violence and divorce rates – these also appear to be reaching “crisis” levels today, and arguably are actually on the rise. However, this could perhaps be a feedback loop created by the existence of myriads of data drowning people every day. “Back in the day”, wives were only told about marriage via the church, elders, and the carefully-curated network television shows on the few channels that existed. Today, the secular world is literally breathing down your neck every second of every day with portrayals of “picture perfect” couples, ideas of what happiness is, what you deserve, etc. Also, many relationships that are fractured with constant fights or abuse now have ample data to support getting a divorce, whereas in the “good ole days”, wives rarely had a way out. (Note: This is not commenting on whether divorce is “right” or “wrong”, but merely that wives (and husbands) have more data now to make decisions than previously.) Gun violence is a similar feedback loop, with more and more people seeing gun violence in their everyday data and increasingly seeing it as a “solution” to their problems.

Climate change is the other example I mentioned – now, I don’t doubt for one second that climate change is real or that mankind may be influencing it. I do claim that the mountains of data we have today versus the vacuum of data centuries ago creates a bias in our views. It seems very likely that the world is warming and that humans may be at least part of the cause; but making extrapolations of data into the past or future based on sparse or weak data and imputing that into trends is a dangerous idea.

I think humanity may not even realize the flood of data that sweeps over them everyday now and how that is influencing their decisions in even subtle ways. It just has never been the case that humans have had access to so much information at every moment of the day – smartphones, TVs, computers, vehicles, and even our homes are becoming “smart” and drowning us with data. Our brains were not meant to handle such overload. Memes, trends, and fads race across the neural world as mental viruses. Just as data scientists are struggling to grapple with a new age of Big Data, each and every one of us are incrementally and rapidly evolving our mind’s framework, one piece of datum at a time…

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: