Main image of article Big Data Means More Need for Hadoop Skills
Big Data is pressuring companies to find more professionals with analytic skills. Madan Sheina, a lead analyst for Ovum’s Information Management Software group, last week posed the question of whether Big Data will create new jobs or require retraining of existing IT staff. How seriously are you taking Big Data? Share your thoughts below.  Jill Dyché, a partner and co-founder of Baseline Consulting, sees a combination:
The jobs that we created to do data governance – like data steward, chief data officer – are more important than ever with Big Data. But there’s also new skills as we look at some of these newer technologies like Hadoop and MapReduce. They’re not your father’s SQL database. “So the DBAs in your organization may or may not have the skills you need to manage and load and store all this data. … The traditional DBA is much more of a management-type role. The new breed of administrators are much more about design and optimization. … It’s a different mindset.
She says database administrators, ranked the second hottest tech career of 2012 by U.S. News and World Report, will continue to be important regardless of whether companies take a new approach to their data. And she sees an increasing need for architecture and systems design skills as companies figure out how to make new technology fit with their existing systems. While open source Hadoop is becoming the infrastructure of choice for many organizations, Bob Mahan, head of training for Hadoop vendor Hortonworks, says even Fortune 1,000 companies are doing more research and testing than are actually using it. Dyché agrees, adding that vendors such as Oracle and Teradata are scrambling to supply support packages as they try to convince customers not to abandon their existing platforms. With his company’s recently launched Hortonworks University, Mahan sees three primary roles: manager, developer and data scientist.
The number of job postings with the word Hadoop has increased 3,000 percent in the past 18 months … but there’s not even that many users out there with even mid-range skills,” he said. He and Dyché agree that companies’ inability to hire this talent will force them to retrain existing IT staff. “Companies are very concerned about retooling their resources.
Hortonworks’ training for Hadoop developers introduces them to aspects such as Pig, Hive, HBase and HCatalog. The manager training includes not only how to create and manage the technology, but also addresses security, availability and single point of failure. Hortonworks plans to begin offering certifications in the next couple of months. Jeetu Patel, chief strategy officer and chief marketing officer for the Information Intelligence Group at EMC, sees data scientist as the key role that has come from big data. With training, he says, business intelligence practitioners can evolve into these new roles that involve the ability to discern patterns and apply data-based knowledge to business. But this function won’t be limited to IT or to designated BI analysts, he said. It will be a function that involves myriad jobs, whether they be in marketing, product development, supply chain management, infrastructure, application development or other units. Traditional BI, he says, relies more on historical data while analytics requires more skill in iterative analysis along with the ability to recognize patterns. Patel said:
In every aspect of IT, you’ll start to see rigor around data. If I look at my list of IT projects, how do I prioritize those projects, not just on who’s screaming the loudest, but on the true business need of those projects? If you’re in networking, and we’re kicking off a sales campaign, there might be some bandwidth-optimization techniques that can be used based on the data. Historically, data scientists have been with companies that are technologically astute. In the future, data scientists are going to be at every kind of company… We believe that data scientists will be sprinkled throughout every function in the organization.
Patel sees the skills and the technology evolving together as new capabilities spark the imagination of workers, who then team with vendors to provide even further capabilities.