Big data has never been bigger, nor more of a crapshoot. At least, that's the sense one gets from a new survey revealing that 76 percent of all enterprises are looking to maintain or increase their investments in big data over the next few years -- despite a mere 23.5 percent owning up to a clear big data strategy.
That wouldn't be so bad if things were getting better, but they're not. Three years ago 64 percent of enterprises told Gartner that they were hopped up on the big data opportunity. But then, as now, the vast majority of big data acolytes didn't have a clue as to how to get value from their info stores.
Despite our best attempts to capture signal from all the big data noise, in other words, we're mostly flying blind.
Bigger and bigger!
The consultancy DNV GL Business Assurance, in partnership with research institute GFK Eurisko, polled 1,189 enterprises across the globe to better understand their big data plans. A majority of these companies -- 52 percent -- see big data as a big opportunity. That number climbs to 70 percent among large companies (1,000-plus employees) and tops 96 percent of those the report authors categorize as Leaders.
This jibes well with the survey's other finding that nearly 45 percent of enterprises see big data as "important" or "very important" to their current operations. It leads, not surprisingly, to big plans to invest even more into big data:
That's the good news.
What am I supposed to do with it all?
The bad news is that despite all this earnest energy to put big data to use, the vast majority of enterprises don't. More troubling still, despite all the enthusiasm for investing more in big data, these same companies haven't shown a particular penchant for effectively planning out how to improve the situation.
Sure, 100 percent of those companies classified as leaders are, by definition, "leveraging big data to boost their productivity and value creation," but 73.2 percent of everyone else is adrift in the big data sea. Even most large enterprises can't seem to buy a big data clue:
Furthermore, the percentage of companies actually deriving value from data is unlikely to rise, given that most companies lack a clear strategy for putting their data to work:
Big companies are a bit further along (39.6 percent of them claim to have a clear strategy), perhaps because they invested in big data earlier than most, but the fact remains: Most companies still don't know what to do with their data.
Again, this fact hasn't changed much in the past few years. As Gartner uncovered in 2013, most companies struggle to define a strategy, determine how to get value from their data, and find the skills necessary to do so. As I wrote then, "When 56 percent of respondents struggle to know how to get value from their data yet are either deploying or planning to imminently deploy a big data project, we have a serious disconnect."
It was true then, and it's true now.
More data, more noise
The problem may come down to Nate Silver's contention in his book, "The Signal and the Noise" -- more data doesn't necessarily equate to more insight. In fact, it often yields the opposite:
If the quantity of information is increasing by 2.5 quintillion bytes per day, the amount of useful information almost certainly isn't. Most of it is just noise, and the noise is increasing faster than the signal. There are so many hypotheses to test, so many data sets to mine -- but a relatively constant amount of objective truth.
Enterprises that jump willy-nilly into the big data fray quickly discover that "big data" often translates to "big confusion," and more data doesn't solve that. It exacerbates the issue.
Back in survey land, most respondents (80.8 percent) to the DNV GL survey acknowledge that they and their management aren't adequately preparing for a big data future. Couple this with a lack of enterprise strategy around data and we have a recipe for big data implosions, not insights.
In response, 47.3 percent of respondents claim to be investing in in-house training to improve their chances of discovering signal in the the ever-increasing noise of their data. This strikes me as the one clear positive from the report. After all, as Gartner analyst Svetlana Sicular noted years ago, "Organizations already have people who know their own data better than mystical data scientists."
The key, she continues, is to enable people who understand the business to also intelligently query the business' data. "Learning Hadoop is easier than learning the company's business," she posits, so more efforts should be made to train existing employees on the tools necessary to unlocking the value of enterprise data.
In sum, we don't need more data -- not really. What we need is more people who can understand the data we already have.