According to this recent survey on enterprise AIOps (Artificial Intelligence for IT Operations) adoption, 67 percent of enterprise IT organizations in the United States have experimented with artificial intelligence (A.I.) and machine learning (ML) for data management and incident remediation. What’s more, research firm Gartner fully expects that artificial intelligence will create more jobs than it replaces by 2020.
A.I. is moving fast and enterprises need talent today. But not just any talent. What once was a shortage of coding and software engineering expertise has now evolved into an overall shortage of skills in artificial intelligence and algorithmic engineering.
No Skills to Pay the Bills
Skills gaps are cited as among the biggest hurdles to AIOps adoption and implementation; a recent EY survey of 200 senior leaders found that 56 percent see talent shortages as the single biggest barrier to implementing A.I. in business operations. It’s clear that finding machine learning engineers is not an easy task; they’re a bit of a unicorn, combining engineering fundamentals with data modeling and statistical analysis. But with the right framework, it’s entirely possible to build a team that has the right mix of data science, engineering know-how, and even a little robot emotional intelligence.
Math in the Machine
To start, machine learning engineers need deep expertise in predictive modeling and statistical analysis. At my company, we look for engineers who can combine core engineering fundamentals with the ability to see patterns in data and translate that into action. Not only do they need to be able to manipulate code and build software, but they must also understand how mathematical models can create insights, and those insights can drive action.
Some practical questions during our initial interview rounds include:
“When should you use classification over regressions?”
Setting up use case-driven questions like this helps us understand how a candidate builds mathematical models. Fundamentally, classification is about predicting a label, and regression is about predicting a quantity.
“Do you have experience with Apache Spark or some public machine learning libraries? (TensorFlow, etc.)”
Ultimately, we want to make software that’s scalable. This is the advantage of a machine learning library. Today’s A.I. engineers must be proficient in modern tools to build enterprise-grade solutions.
Power to the Problem
The next challenge for a true A.I. engineer is solving the big problem of clean data. Creating datasets that are rich, contextual and clean provides the best results for an artificial intelligence solution, as AI relies on data for decision-making. Some questions may include:
“How would you handle an imbalanced dataset? How do you handle missing or corrupted data in a dataset?”
These questions get at the importance of building models using clean data. At its core, A.I. must use clean datasets for insights. Otherwise, the actions will be incorrect or useless.
“Provide an example of why you would use quicksort versus binary.”
These are both algorithmic options for organizing the data, and help illustrate a use case of two extremes. An A.I. engineer can employ either one.
“How do you clean and prepare data to ensure quality and relevance?”
This question is relatively tactical, but helps us understand the critical processes that the engineer employs at the most critical points in A.I. engineering.
In short, I look for engineers who can apply statistical analysis to data for enhancement, cleaning and processing, but who also understand the practical ramifications of data as the engine of software.
At its core, machine learning engineering sounds like a natural evolution in software engineering. Before, we were looking for engineers who could translate basic human requests into some sort of computer-based action; now we’re simply trying to anticipate those human requests. The basic mechanics of data ingestion, analysis, interpretation and action are the kind of actions that humans take every day; turning these steps into an action that a machine can take unsupervised is an entirely new challenge.
This is where statistical analysis and predictive modeling come into play. The true machine learning engineer is both a geek and a craftsperson; she’s a math nerd and a builder. Our talent search helps us identify candidates who live at this intersection, and who can help us continue to win the race for efficient, effective technology.
Bhanu Singh is the Vice President of Engineering for OpsRamp.