Thanks to the shortage in data-science talent, more organizations are relying on analytics-as-a-service offerings. This trend has set off something of an arms race between traditional providers of IT services and cloud-based vendors.
Cloud-based analytics services live and die by the data and tools they offer. That’s part of the reason why IBM, which wants to become the biggest player in the analytics space, acquired of the digital assets of The Weather Company.
Speaking at the recent IBM Interconnect 2016 conference, The Weather Company CTO Bryson Koehler said becoming part of IBM could advance The Weather Company’s efforts to enable organizations to incorporate its data into analytics applications. The Weather Company also wants organizations to use custom-built tools and APIs to analyze weather data.
“Weather is the original Big Data problem,” Koehler said. “We make 26 billion forecasts every day and process 348,000 API calls every second.”
All that information is stored in the (digital) cloud. IBM exposes that data to its Watson analytics service, which aims to improve the accuracy of weather forecasts as well as the impact the weather might have on businesses.
Koehler asserts that, on a three-day basis, The Weather Company is 78 percent accurate on its forecasts. The goal, said Koehler, is to improve that accuracy of those forecasts by collecting more data via connected vehicles of all kinds, smartphones and even traffic lights; in effect, the Internet of Things (IoT) will provide the data sources needed to more accurately track and eventually predict what’s occurring in the atmosphere.
The Weather Company is not the only source of data about the weather. But as companies start to better understand the impact weather has on everything from retail store sales to the amount of oil pumped out of the ground, the more valuable that data becomes. That’s why IBM not only acquired The Weather Company, but plans on making its data available via the analytics services it hosts on the IBM SoftLayer cloud.
Judith Hurwitz, principal analyst for consulting firm Hurwitz & Associates, says it’s clear that organizations of all sizes are now in a race to turn data into actionable intelligence.
The challenge they all face is that, outside of government agencies, there isn’t a lot of relevant data residing in one place; entities such as The Weather Company are the exception rather than the rule when it comes to Big Data. Organizations need to figure out how to aggregate hundreds of sources of external data in a way that enables them to correlate that information against their own internal data.
In many instances, this requirement is creating new roles such as Chief Data Officers. But because most organizations lack the internal expertise to acquire and manage Big Data, Hurwitz thinks there’s going to be more reliance on information-services providers to accomplish many analytics tasks.
“There’s a lot of good reasons not to do this yourself besides not actually having the expertise,” Hurwitz said. “You may not want to go to the cost and trouble of owning the data. Not everybody needs to have access to certain types of data every hour of the day.”
A lot of organizations may fall into the “rent” rather than “own” data camp. But demand for data scientists won’t slacken as a result; rather, it means that firms that create business models around data will attract top talent. That could make it difficult for some organizations to source tech pros skilled in data science, pushing them into the arms of information-service providers.
Vendors such as IBM hope that hosting as much data in the cloud as possible will serve as a catalyst for developers to host more applications on their respective clouds; although developers often seek to host applications as close as possible to where data resides, in hope that it will boost overall application performance (a phenomenon known as data gravity), those vendors believe they can craft data-as-a-service strategies that will please this segment.
Of course, collecting data and paying lots of money for data scientists (as well as analytics tools) doesn’t guarantee ultimate business success. But it does tend to give organizations a competitive advantage.