Back in August, The New York Times asked if so-called Big Data was “an economic big dud.” The reasoning went something like this: despite massive growth in the amount of data held by individuals and corporations, economic productivity has slowed over the past few years—suggesting that insights into data aren’t actually translating into increased efficiencies.
“Those factors have some economists questioning whether Big Data will ever have the impact of the first Internet wave, let alone the industrial revolutions of past centuries,” that article surmised. “One theory holds that the Big Data industry is thriving more by cannibalizing existing businesses in the competition for customers than by creating fundamentally new opportunities.”
Some economists and analysts disagree with that assertion, of course. They argue that data analytics hasn’t yet penetrated society to its maximum extent, and that the nebulous nature of data—it’s not a physical commodity in the same way as electricity or gasoline—makes it hard to gauge ultimate impact.
Those advocates can point to crowdsourcing, IBM’s Watson (particularly its growing work with oncology), the backend infrastructure powering Facebook and Netflix, and a couple other projects as prime examples of “Big Data” at work. But for tens of thousands of smaller firms, the benefits of data analytics are often less clear—and those companies face the stark decision of whether to spend considerable resources on a costly analytics package, or risk falling behind competitors.
Earlier this year, research firm Gartner suggested that Big Data is plunging into a “trough of disillusionment” that could dissuade many such firms from buying into analytics technology. That trough is part of Gartner’s regularly updated Hype Cycle, in which all technologies—cloud computing, tablets, whatever—progress from rising interest to overexposure to general disillusionment to, at last, rehabilitation as productive and well-integrated technologies.
Dealing with massive datasets and complicated software is hard, Gartner analyst Svetlana Sicular suggested at the time, which is driving companies to re-evaluate how much they get out of such solutions. “Validating answers is also a tough job,” she added. “Big data analytics deals with uncertainty: you do not deduct the number and say that the meaning of life is 42—you get a proof of your hypothesis with a certain degree of confidence.”
But if “Big Data” wants to continue its momentum in 2014, it will need to more clearly demonstrate that it can pay off in a big way for firms large and small. That means more clear-cut “victories,” in which companies demonstrate how they managed to dominate a market segment or make a ton of money via analytics. Given the fragmentation in the data industry as a whole, it could be difficult for such “wins” to stand out; but without them, “Big Data” risks being seen as a lot of empty hype.