Will Intel’s Hadoop Distribution Increase Big Data’s Hype?

On the same day Intel announced it would launch its own Apache Hadoop distribution (which will take full advantage of its hardware), Pentaho revealed that it had entered into an OEM licensing agreement with the chipmaker. Apache Hadoop is an open-source framework increasingly favored by companies that need to crunch large amounts of data on large hardware clusters.

Under the terms of that agreement, the Intel Distribution for Apache Hadoop will feature a host of baked-in Pentaho software applications, including data-mining and predictive-analytics packages.

This isn’t the first time that Pentaho’s partnered with a large IT vendor for some sort of integrated service: back in September, it joined forces with Cisco to bring its data-analytics software to the latter’s Unified Computing System. Before the end of last year, the company also made significant upgrades to its Business Analytics and Mobile BI tools.

But given the prevalence of Intel chips, not to mention the rising popularity of Big Data software, it’s tempting to see Pentaho’s alliance as the biggest of its recent announcements. There’s also the possibility that Intel’s move to bake data-analytics software into its hardware will kick off a new stage in Big Data’s rapid (and increasingly hype-filled) evolution, one in which hardware vendors and software builders team up in greater numbers to offer “complete packages” for data crunching. If Intel is teaming up with a significant data-analytics firm, what’s to stop an AMD or other ARM-based manufacturer from signing on the dotted line with another Big Data purveyor?

The aforementioned hype, that’s what. Analysts generally seem to agree that Hadoop-related products will remain popular over the next several years. IDC, for example, thinks the Hadoop market will hit $812.8 million by 2016—somewhat conservative compared to other estimates, which place that number in the low billions over the next few years.

However, Apache Hadoop is also open-source, meaning that IT vendors who hope to profit from their own commercialized take on the platform will need to compete against a variety of free and low-cost offerings pushed into the ecosystem by pretty much any company or developer with the skills. Some months ago, researchers at IDC theorized that competition between open-source and commercial Hadoop platforms could force vendors producing the latter to lower their fees in order to compete, which would reduce revenues. That would perhaps cool the hype a bit.


Image: IdeaStepConceptStock/Shutterstock.com