Big Data Jobs: Big Demand, for Those with the Skills

Not all data jobs are the same. People talk about Big Data and data science as if they’re catch-all phrases requiring a particular, monolithic skill-set. They’re not: data-related jobs occupy a broad spectrum, and the person who suits one job may not suit another.

If you’re wondering where you fit in the data universe, quantitative assessment firm Correlation One has developed a helpful mapping methodology that explains precisely what each role involves and the skills required.

What’s a Data Scientist?

A data scientist is a kind of catch-all description for someone who finds, processes and analyzes data in order to extract meaningful results that can predict the future. As a data scientist, you’ll need to be able to devise ways to collect data, and you’ll also need to be able to interpret that data that’s delivered. You’ll also need to be particularly good at developing and testing hypotheses.

What’s a Data Engineer? 

As a data engineer, you’re going to be much more about ‘data wrangling’ or preparing the data for use and then devising ways of presenting and analyzing that data.

You’ll need to be familiar with SQL and database programs, and you’ll also need to know how to write algorithms that can parse the data. You’ll need to know Extract, Transform and Load (ETL) processes, and you’ll need to understand how to structure data in hierarchies and create visualization tools that allow users to access and interpret the data more easily.

What’s a Quantitative Researcher?

If you’re a quantitative researcher, you’re mostly going to be about investigating the data (including through visualizations and hypothesis testing, again) and using the data to make predictions about the future. To make predictions, you’ll need to understand quantitative modeling techniques and model validation, including the dangers of under- and over-fitting your model to the data available.

What’s a Machine Learning Researcher?

Machine learning researchers have a whole category of their own, according to Correlation One. Their main purpose is automating the use of data to make predictions about the future. 

What’s a Business Intelligence (BI) Analyst?

B.I. analysts work with end users in the business to look at what historic data says about a business’s performance. B.I. is less about predicting the future and more about analyzing what the past says about the present.

If you want to work in B.I., you’ll need to be excellent at sourcing, describing and synthesizing data. You’ll need to be able to develop and test hypotheses and to communicate them to the business. And you’ll need to know a lot about that business (which isn’t always imperative for pure data roles) for your conclusions to be meaningful.

What’s a Database Developer?

If you’re a database developer you’re going to be at the forefront of data storage and retrieval. You’ll need excellent knowledge of SQL and systems such as Kubernetes

What’s a Product Manager?

Lastly, a product manager works closely with end users (e.g., portfolio managers in a hedge fund) to devise a data-led product that will be valuable to the business. In the words of hedge fund Two Sigma: “Product managers aim to create products that customers love.” They are also “the conduit between the users, the business, and the engineers” who are “held accountable for the end-to-end user scenarios.”

If you want to be a product manager, you’re going to need to know a lot about the business you’re working for. You’ll also need to understand how to generate, interpret and communicate hypotheses resulting from the data, and how to source data in the first place. 

Data Jobs Require a Bit of Everything

The charts below from Correlation One reflect what’s required for different jobs in Big Data. Dark blocks (number 5 on the color code) are the skills that are imperative for each role; light blocks are the skills that help but aren’t essential. 

Most jobs require a few specialist skills, plus a bit of everything. And as Correlation One points out, different employers use job titles to refer to different things. For instance, a data scientist at one place might be called a business analyst at another. Check out Correlation One’s chart below!

This article originally appeared in eFinancialCareers.

One Response to “Big Data Jobs: Big Demand, for Those with the Skills”

  1. Big Data, all the rage in California Healthcare, is already done growing. Unless you work for Google or Apple it is really a ‘useless skill to attain unless you live in a metropolitan area that has a startup that doesn’t realize big data is useless’ to pay you.