Niantic Labs, made popular by the blockbuster game "Pokemon Go," has unveiled a new developer platform for augmented reality (AR) developers. The aptly named ‘Niantic Developer’ portal sits quietly as a page where interested developers can submit their email, but an accompanying blog post provides a lot more detail on Niantic’s mission. Dubbed the "Real World Platform," Niantic’s developer tooling will allow anyone to build "Pokemon Go"-style games, only better. Niantic says it will "advance" how computers see the real world, “moving from a model centered around roads and cars to a world centered around people.” The plan is to use machine learning to decipher chairs, pathways, roads, bushes, and even people. Today, "Pokemon Go" uses publicly available databases as a means to know which Pokemon belong in which environments (so you don’t catch a water Pokemon in the desert). When the platform evolves, Niantic can use machine learning to ‘see’ that a giant fountain in otherwise-desert-y Palm Springs is also water, and that it might be an appropriate place for a water Pokemon. This new software will allow characters and objects to run around a scene, jumping atop chairs and tables or hiding behind objects; the Real World Platform will understand the “meaning” to your surroundings, at least in the context of AR. (That water Pokemon might jump out of the water and onto a nearby table, for instance.) Multi-player support is also coming, and it looks similar to what Apple recently announced at WWDC with ARKit 2. Niantic's most intriguing demo is a multi-player puzzle game reminiscent of Monument Valley, or Myst: All of this seems possible in no small part due to an acquisition of London-based Matrix Mill. From Niantic’s announcement:

The Matrix Mill team comes from University College London, where they spent years building and perfecting deep neural networks that can infer information about the surrounding world from one or more cameras. This technology redefines how machines see and understand the 3D world, and more importantly, how digital objects can interact with the real elements of it. At Niantic, we frequently talk about how in order to augment reality, you need to be able to understand it. The Matrix Mill team has come up with novel ideas that push the boundaries of what machines can process, thinking around occlusions, and seeing the world closer to the way human eyes can. As a result of this hard work, AR experiences can feel more natural to the eye, which is a goal we have squarely in our sights.

Niantic’s developer platform might not be ready for the real world (yet), but it has distinct advantages. There are already proof-of-concept demos (as seen in the video above; there are more on the Niantic blog post), and a massive mobile device install base. Magic Leap’s SDK promises much of the same technology Niantic now does, but requires a still-unreleased proprietary headset. Niantic chief John Hanke says he wants the Real World Platform to be like AWS for augmented reality apps. That’s a lofty goal, but his company is already closer to that milestone than anyone outside of Apple, Google or Microsoft.