In a recent posting on Medium, Robert Chang, a data scientist at Twitter, described what it’s like to mine data for the social network. In the course of doing so, he offers some interesting insight into the differences in analyzing data for startups versus larger enterprises.
“One important lesson I learned at Twitter is that a Data Scientist’s capability to extract value from data is largely coupled with the maturity of the data platform of its company,” he wrote. “Understand what kind of DS work you want to get involved, and do your research to evaluate if the company’s infrastructure can support your goal.”
In his estimation, data scientists at startups focus more on foundation-building tasks such as implementing logging and building ETL processes; once the company grows in size (if it survives, that is), the scientist can devote more time to actually mining insights from data. Once a company becomes huge, scientists need to make data operations more efficient and powerful.
The rest of Chang’s post is worth reading for insight into the skills and experience necessary to analyze data at a fast-moving company like Twitter. But how does one become a data scientist in the first place? Over the past few years, multiple analyst firms have concluded that the country faces a severe shortage in the number of people capable of crunching data, so anyone willing to learn the skills and degrees can likely land an interesting position somewhere.
According to Simon Hughes, chief data scientist on the Dice Data Science Team, getting a master’s degree in some aspect of data science, predictive analytics, machine learning, or statistics is a good way to break into the industry. There are “data science boot camps” where you can learn necessary skills. Employers hungry for data scientists are also increasingly willing to train new employees in the nuances of analyzing data for insights.
Once you learn the skills and earn any necessary degrees, you should also ask yourself what kind of company you want to work for. A startup offers the opportunity to build an analytics platform from the ground up… but a larger company can give you the chance to do some very interesting, cutting-edge things with a massive reservoir of data.