Apple has struggled in recent years to establish a robust artificial intelligence (A.I.) practice. This partially stems from the company’s ironclad privacy policies—it’s more difficult to analyze datasets for insights when internal rules prevent the company from using every piece of user data it can vacuum up. Nonetheless, Apple’s newest projects show that it’s powering ahead anyway—including one platform that, if it’s ever released, could change how you use A.I. and machine learning (ML).
(It’s worth remembering how, in a 2015 speech, Apple CEO Tim Cook accused tech giants such as Facebook and Google of “gobbling up everything they can learn about you and trying to monetize it,” which he framed as “wrong.” It seems unlikely that Apple’s stance on data and privacy will change during Cook’s tenure.)
According to a just-released paper with the dry-but-mysteriously-compelling title “Overton: A Data System for Monitoring and Improving Machine Learned Products,” a group of Apple researchers describe their work on a machine-learning platform (named—you guessed it—“Overton”) designed to “support engineers in building, monitoring, and improving production machine learning systems.”
How does Overton go about this herculean task? By automating the nitty-gritty of machine-learning model construction, deployment, and monitoring. Apple claims that the platform is already in use, supporting multiple initiatives “in both near-real-time applications and back-of-house processing.” These Overton-powered applications have “answered billions of queries in multiple languages and processed trillions of records reducing errors 1.7 – 2.9x versus production systems.”
This means that any researcher or engineer working with Overton will need to trust that the platform can recognize and fix issues with a model; otherwise they’ll presumably need to dig into the algorithms and datasets themselves, a lengthy and stressful process. But if it truly works as it says on the proverbial tin, it should reduce the time necessary to churn out results. Here’s an except from the paper on what the model inputs:
Overton takes as input a schema whose design goal is to support rich applications from modeling to automatic deployment. In more detail, the schema has two elements: (1) data payloads similar to a relational schema, which describe the input data, and (2) model tasks, which describe the tasks that need to be accomplished. The schema defines the input, output, and coarse-grained data flow of a deep learning model. Informally, the schema defines what the model computes but not how the model computes it: Overton does not prescribe architectural details of the underlying model (e.g., Overton is free to embed sentences using an LSTM or a Transformer) or hyperparameters, like hidden state size.
And this is what Overton does with that schema/input:
Given a schema and a data file, Overton is responsible to instantiate and train a model, combine supervision, select the model’s hyperparameters, and produce a production-ready binary. Overton compiles the schema into a (parameterized) TensorFlow or PyTorch program, and performs an architecture and hyperparameter search. A benefit of this compilation approach is that Overton can use standard toolkits to monitor training (TensorBoard equivalents) and to meet service-level agreements (Profilers). The models and metadata are written to an S3-like data store that is accessible from the production infrastructure. This has enabled model retraining and deployment to be nearly automatic, allowing teams to ship products more quickly.
So Overton is going to reduce the amount of coding that machine-learning researchers and data scientists need to do—allowing them to observe and manage the process from a higher level. Plus, it’s interoperable with platforms such as Google’s TensorFlow, which are becoming industry-standard.
Apple isn’t unique in producing a tool that attempts to take as much of the coding grind out of the machine-learning process as possible. For example, Google has AutoML, which is similarly designed to produce working machine-learning models with a minimum of code; there’s also Microsoft’s Machine Learning Studio, which attempts to boil down ML model building to a drag-and-drop process. Automating ML and A.I. is key to these technologies going as mainstream as possible.
The revelation of Overton is also interesting, as it shows that Apple’s researchers are moving on parallel tracks to other tech firms. Apple’s tool might help its internal staffers catch up to their rivals in A.I./ML, but it’s an open question whether they’ll ever transform it into a public-facing product, just as they've done for CoreML and other tools.
