Big data is less about technology and more about organizational mind-set and a new form of practical business savvy. That is not the way most businesses are seeing it, and vast sums are being spent on hardware and software that may never be used.
From our work with companies in Australia and internationally, we see five obstacles. First, most companies do not have a sound grasp of data they have had for years. They don’t make use of the “small” data directly provided by their customers and staff and rely instead on entrenched wisdom, anecdote, and assumption. A basic curiosity about how their organization and industry really works is essential for data analytics to make a difference.
Second, where companies have scaled up their analytic ambition, many are hitting roadblocks when it comes to acting on the insights. Successful Internet companies such as Google, Facebook, Amazon, and Netflix generate every move from these insights.
But in the mainstream, many businesses still run on the gut instinct of the chief executive, who is not comfortable with such a powerful and unpredictable challenge to his or her wisdom. Embracing big data requires a fundamental change in orientation for most companies.
Third, big data done well can yield an accelerating stream of highly specific new growth opportunities. Businesses are often not ready for this, even if the chief executive is onside. They are set up to execute a small number of big-picture strategies over months and even years. Big data usually yields many smaller ideas requiring a highly efficient execution machine. Insights must be put into practice before competitors copy them or customer demand wanes.
The fourth issue concerns the hype around big data and the pressure on boards and chief executives to “do something.”
The need for action is commonly delegated to CIOs, who pull the levers over which they have control: they build data warehouses, initiate data cleansing and architecture improvement, and deal with data governance. But the underpinnings of big data are not in these areas. It can thrive on highly cost-effective cloud services and diverse and often “ownerless” data sources. Terabytes of data can be stored for as little as $100 on a pay-per-use basis.
Tackling big data with traditional enterprise IT approaches is expensive and inevitably leads to dashed expectations.
The fifth major barrier arises when companies that have overcome the above obstacles try to scale up their isolated successes. When you have so much data and potential in front of you, where do you start?
This is the bottleneck in the big-data revolution—a global shortage of skilled data navigators who can act as a bridge between the thousands of questions that businesses are asking (or should ask) on the one hand, and the potential of analysis to answer them on the other. Big data might tell your marketing manager an above-average number of cinema goers on Tuesday who saw the latest X-Men ate a Mars bar for breakfast, but how does that help you make money?
The Googles of this world prove the power of big data, but it requires much more than ingenious algorithms, technology spending, and a passing interest. Chief executives and their boards need to understand the business-changing implications of analytics in their businesses and only then commence spending.
This commentary was originally published in The Australian.