Google’s A.I.-Powered AR Microscope Tech Could Fight Cancer

Over the past few years, a number of tech giants have focused their artificial intelligence (A.I.) and machine-learning capabilities on curing cancer. For example, IBM markets its Watson platform to hospitals as an ideal tool for processing and analyzing oncology data. And Google has tested a machine-learning algorithm capable of predicting 92.4 percent of tumors in a sampling of medical slides.

Now Google is utilizing another type of cutting-edge technology to fight cancer: augmented reality. Specifically, the company has brainstormed a prototype Augmented Reality Microscope (ARM) that leverages the company’s existing work in deep learning to analyze biological samples.

“The platform consists of a modified light microscope that enables real-time image analysis and presentation of the results of machine learning algorithms directly into the field of view,” Google wrote in a blog posting about the technology. “Importantly, the ARM can be retrofitted into existing light microscopes found in hospitals and clinics around the world using low-cost, readily-available components, and without the need for whole slide digital versions of the tissue being analyzed.”

Perhaps inevitably, this ARM platform will support deep-learning models developed in TensorFlow, Google’s architecture for that technology. For those who are new to the wonderful world of A.I., TensorFlow includes an open-source (under Apache 2.0) software library for numerical computation using data-flow graphs; after building a graph, a tech pro or researcher can write loops that power and refine computations. (And if you’re really new to machine learning and A.I., Google also offers an “Intro to A.I.” course with 25 online lessons and more than 40 exercises, which theoretically can be completed in 15 hours or so.)

“As in a traditional analog microscope, the user views the sample through the eyepiece,” is how Google explains the hardware’s operation. “A machine learning algorithm projects its output back into the optical path in real-time. This digital projection is visually superimposed on the original (analog) image of the specimen to assist the viewer in localizing or quantifying features of interest.” The digital projection will continue to seamlessly align with the sample even if the researcher changes magnification or adjusts the slide.

Possible AR imagery will include text, arrows, animations, and heat-maps; the platform could outline or point to possible tumor sites. Of course, the applications go beyond cancer: any type of pathology discoverable via a microscope, whether tuberculosis or malaria, is a potential target for this technology.

For tech pros interested in working in medical technology, artificial intelligence will almost certainly have a significant effect on healthcare in coming years. That being said, the technology is still nascent, and various healthcare regulations could slow its adoption.