It’s been a big year for Big Data: more companies than ever invested in data-analytics systems, hoping to gain valuable insight into their operations. At the same time, IT vendors large and small issued a variety of new platforms designed to meet those analytics needs. Many of the trends that dominated 2012 will almost certainly continue in 2013:
Apache Hadoop
Although the open-source Apache Hadoop framework has been around for a few years, it gained sizable traction in 2012. A number of large companies turned to Hadoop to help run massive data applications on large hardware clusters; meanwhile, various IT vendors integrated Hadoop support into their analytics and storage offerings. Over the summer, a report from Market Research Media suggested that the spike of interest in Hadoop could continue over the next several years,
transforming into a $2.2 billion market by 2018 (other research firms have pegged the market at a somewhat lower valuation—IDC, for example, believes it’ll hit $812.8 million by 2016). However, Hadoop’s open nature could hinder those IT vendors seeking to make some cash off proprietary platforms. IDC believes that competition between open-source and commercial platforms could force those vendors to lower product-licensing fees, which would reduce revenues. “The Hadoop and MapReduce market will likely develop along the lines established by the development of the Linux ecosystem,” Dan Vesset, vice president of Business Analytics Solutions for IDC, wrote in a statement back in May. “Over the next decade, much of the revenue will be accrued by hardware, applications, and application development and deployment software vendors.” It remains to be seen whether such a trend begins in 2013.
Build Your Own Massive Data System
In 2011, Facebook’s engineers realized their MapReduce implementation from Apache Hadoop was no longer enough to handle all the user data pouring into their infrastructure. Faced with the need for a platform capable of massive scalability and low latency, Facebook did what any multibillion-dollar organization with hundreds of in-house geniuses would do: built their own. The resulting framework, known as “Corona,” supposedly offers superior cluster utilization and job latency to pre-built platforms. While other organizations don’t have Facebook’s resources, they do have a need to wrangle increasing amounts of data. That could drive many of them, over the next year or so, to opt for custom-built solutions over “off the shelf” platforms.
Mobile Apps
Over the past few years, mobile devices have increasingly eclipsed PCs as most users’ computing tool of choice. Technology executives take the stage at conferences to proclaim that, within a few years, a healthy percentage of workers will conduct nearly all their daily business from a smartphone or tablet. Whether or not that vision comes to pass, it’s indisputable that handheld touch-screens are driving much of the technology industry at the moment—which creates sizable opportunities, as well as a host of challenges, for IT vendors. On the backend side of things, organizations that store and analyze data need to build up their infrastructure to handle a growing flood of information from millions of mobile devices. At the same time, IT vendors that build analytics software need to create apps and dashboards capable of displaying data in an optimal way on a very tiny screen. Mobile devices also create security concerns—imagine the repercussions if a tablet holding gigabytes of proprietary data, or one-tap access to a proprietary database, ends up stolen or hacked. And if that wasn’t enough, mobile devices’ reliance on the cloud and near-continuous uptime is another pressing issue. All these issues—
and more—will come to the fore in 2013.
Help Wanted
In early December, IDC issued a report suggesting that investment in Big Data will grow to nearly $10 billion in 2013. Mobile devices, and increased use of Software-as-a-Service (SaaS) and Platform-as-a-Service (PaaS), also topped the research firm’s list of future trends. “Here, converged systems (combining server, storage and network systems together with the software to manage them) and software-defined networking will transition from market hype to market reality,” read its note on the matter, “with enterprise datacenter and cloud-provider user cases coming to market and getting deployed.” But increased investments in Big Data means a need for more data analysts—and those are in short supply. That’s been an issue for some time; back in 2011, a report by McKinsey & Company’s Business Technology Office found that the need for data-analytics talent would exceed supply by 50 to 60 percent by 2018. “There will be a shortage of talent necessary for organizations to take advantage of big data,” read that report. “The United States alone faces a shortage of 140,000 to 190,000 people with deep analytical skills as well as 1.5 million managers and analysts to analyze big data and make decisions based on their findings.” As recently as this October, research firm Gartner cited a need for more people skilled in managing and analyzing data. “The demand for big data is growing, and enterprises will need to reassess their competencies and skills to respond to this opportunity,”
read its research note, which made several other predictions about technology in the near future. “Jobs that are filled will result in real financial and competitive benefits for organizations.” In 2013, count on that increase in Big Data investments, even as the demand for skilled data analysts continues to rise.
Image: Nan728/Shutterstock.com