Apple May Design Chipsets with AR in Mind

Apple AR
Apple campus in Cupertino.

Ahead of its WWDC 2017 developer conference, Apple is making headlines for initiatives that won’t pay off for years. In addition to scuttlebutt that the company is set to make its own GPUs, there are now rumors it is interested in developing and manufacturing proprietary power management chipsets for mobile devices.

According to Reuters, Apple has been “poaching like crazy” from Dialog Semiconductor, the company it currently relies on for power management chips. Those chips control how much battery power is fed to your device, and when. It’s not solely responsible for Facebook draining your phone (that’s on you, and the developers who make the giant Facebook app), but the chipset is still responsible for making sure things like background processes don’t kill your iPhone when not in use.

Dialog can attribute more than 70 percent of its sales in 2016 to Apple, some analysts assert. That’s why the stock is now down 36 percent. Financial analysis firm Barkhous Lampe recently downgraded Dialog stock from “hold” to “sell” on the back of unnamed industry sources whispering news of Apple’s plans. “In our view, there is strong evidence that Apple is developing its own PMIC [power management chipset] and intends to replace the chip made by Dialog at least in part,” said Bankhaus analyst Karsten Iltgen.

This comes days after industry scuttlebutt claimed Apple was working on its own line of GPUs for mobile devices, specifically the iPhone. Imagination Technologies Group issued a press release noting it was throttling down its relationship with Apple, which it expected to end in 15-24 months’ time:

Apple has used Imagination’s technology and intellectual property for many years. It has formed the basis of Graphics Processor Units (“GPUs”) in Apple’s phones, tablets, iPods, TVs and watches. Apple has asserted that it has been working on a separate, independent graphics design in order to control its products and will be reducing its future reliance on Imagination’s technology.

The twist is that Imagination doesn’t think Apple can make its own GPU without infringing on at least some of Imagination’s intellectual property. The two companies have a relationship dating back to the first iPhone.

Apple WWDC 2015

What is Apple Doing?

If we assume these two news items are the tip of the iceberg, it seems as though Apple is taking full control of its supply chain and will at some point make all of its own components: GPU, power management, displays – you name it. If two partners are at risk of losing Apple’s business, so are the rest.

Unfortunately, that means a significant downturn in business for partners. Apple accounts for a majority of Imagination and Dialog’s income; even larger partners such as Samsung would take a significant hit if Apple stopped doing business with them. Still, a wholly contained and managed supply chain (replete with responsibly-sourced packaging) would be something to behold.

At the same time, a power management chipset along with the freshly designed GPU point to the next big thing for Apple: augmented reality (AR).

Apple began laying the groundwork for better native graphical performance on iOS two years ago with the introduction of Metal, its framework for accessing a GPU. Here’s how Apple describes it:

The Metal Framework provides extremely low-overhead access to the GPU, enabling incredibly high performance for your sophisticated graphics rendering and computational tasks. Metal eliminates many performance bottlenecks—such as costly state validation—that are found in traditional graphics APIs. Metal moves all expensive state translation and compilation operations out of the critical path of your most performance sensitive rendering code. It provides precompiled shaders, state objects, and explicit command scheduling to ensure that your application achieves the highest possible performance and efficiency.

Graphics, compute, and blit commands can be used together seamlessly and efficiently. Metal is specifically designed to exploit modern architectural considerations, such as multiprocessing and shared memory, to make it easy to parallelize the creation of GPU commands.

Apple CEO Tim Cook began pointing to augmented reality last year when he said the company was “high on AR for the long run.” He pointed to the iOS game “Pokemon Go,” which layers holographic creatures on an iPhone user’s real-world surroundings, as proof that people are fond of AR in the right circumstances.

Last month, Bloomberg reported the company had “hundreds of engineers” working on its AR initiatives. Staffed with experienced leaders, the team is reportedly working on AR across Apple’s mobile platforms, and considering a headset for hands-free AR games and apps.

Apple Metal WWDC 2015
Apple Metal WWDC 2015

Apple, AR and Developers

WWDC 2017 is the earliest we should see mobile AR news – if there is any. Until then, expect the company to stay mum on AR, as it potentially affects the developer world in a significant way.

It’s not clear if Apple will position AR as a standalone framework, or use it to bolster Metal. Metal’s current design is meant for 3D graphics, which suits AR just fine for some use-cases, but isn’t relevant to all AR situations. “Pokemon Go” is still our best proof-of-concept; although it overlaid characters on real-world images captured with the iPhone’s camera, it kept true 3D rendering out of the mix.

It’s not clear why the app didn’t have more 3D rendering in AR scenes, though a likely scenario involves GPU and battery concerns. “Pokemon Go” was an instant hit, and instantly caused users to snap up battery packs so they could keep catchin’ ’em all. While iPhone GPUs are nothing to scoff at, AR at scale is more powerful than any mobile GPU can adequately handle right now.

Also, controlling the graphical hardware and software stack would be in Apple’s best interest for AR. We’ve previously pointed out some Metal features that may be critical to AR, leading us to believe Apple will keep its graphical framework in-house.

We also may not see AR for iOS any time soon. Reports of power management and GPU chipsets both point to a 2018-2019 release for the hardware. If Apple is waiting to announce until its proprietary hardware solutions arrive, WWDC 2017 could end up being pretty benign. Still, it’s a good time to start thinking AR, because that’s exactly where things are headed.