This blog post was originally published at CEVA's website. It is reprinted here with the permission of CEVA.
A few months ago, I wrote a post about Augmented Reality (AR) and Virtual Reality (VR) devices showcased at the MWC in Barcelona. While there was a lot of hype around AR and VR at many booths, I felt that there was still something missing before this technology could really take off. The main drawbacks that I saw were dependence on external processing power, and short battery life, both resulting in tethered devices. I was curious to see what breakthrough would bring this technology to the mass market, and open the gateway for all its potential uses to the vast public. Over the past few weeks, I suddenly found myself doing a lot of walking and driving around with my kids. You guessed it, I am referring to Pokémon Go. This successful game, which allows users to capture, train, and battle virtual creatures in real life surroundings, emphasizes the above points.
Where other AR/VR apps failed, Pokémon Go succeeded
Over the past few years, billions of dollars have been poured into AR/VR development. All the tech giants have been trying to develop the software, hardware, and packaging to become the next big thing. What finally did the trick is a very simple use of overlaying computer graphics on the actual surroundings using a camera and GPS. The simplicity is actually what made this such a hit. There is no need to buy any additional hardware, or bulky headsets, or even a cardboard box that folds into a phone cradle. Just download an app to your smartphone, and voila, you are using augmented reality. The ubiquity of smartphones already ensures a vast pool of potential users. Other AR/VR applications have limited themselves to the relatively small gaming community due to expensive equipment, and the time required to be fully immersed in them. This game comes along and lets anyone with a smartphone play, anywhere they happen to be. Of course, the simplicity of the application doesn’t mean it was easy to design and make. The key to its success is being fun and engaging, and having mass appeal. But the fact that it can be downloaded and played by billions of smartphone users and its innovative functionality, is what opened up the possibility of it going viral, as it did.
Just what kind of reality is this, Augmented, Virtual or Mixed?
To understand where Pokémon Go falls in the categories of AR, VR, and MR (mixed reality), you should check out this post on TechCrunch. It includes the distinctions between each one, as well as the different levels of immersion. I also totally agree with the bottom line of that post: “the public thinks Pokémon Go is AR. So it is.” Whether or not it fits the exact definition of virtual, augmented, or mixed reality isn’t as important as the fact that it is perceived as such. This perception means that the public is ready and willing to take on the potential of these technologies, if they are engaging enough and packaged correctly. This may not change the amount of money being invested in the AR/VR industry, but it will surely change the focus of application development, aiming at making them more accessible to more users.
Battery Low: Draining the fun out of Pokémon Go
One of the biggest issues with the game is that it really drains the battery. I’ve felt this first hand, as all of a sudden my smartphone has become a main attraction to my kids, forcing us to travel with power banks wherever we go. Real evidence is also seen by the surge in smartphone backup batteries. Using backup batteries doesn’t seem like a very sustainable solution, both economically (for the users) and convenience-wise.
Running low on Pokéballs? Please connect to charger
The makers of the game are also planning to release a peripheral gadget to tackle the issue of battery usage, called Pokémon Go Plus. The Pokémon Go Plus is a small device that connects to the user’s smartphone via Bluetooth Low Energy (BLE), and uses LED lights and vibrations to notify them of events in the game, reducing the time they need to keep their eyes glued to the screen. But this solution is also not ideal, for a few reasons. First of all, it spoils the great advantage of availability of the game, by requiring extra hardware. Second, the visual content is what makes the game appealing and also what allows the developers to generate revenue. Finally, the average user can’t be expected to buy a peripheral gadget for every new AR game or app that comes out. And I’m sure we can expect to see quite a lot of these in the near future.
The solution for this has to come in the form of more efficient processing, and less waste of energy. As applications become more complex and incorporate more sophisticated features, power consumption will become even more critical. What complex and sophisticated features you ask? I’ll sum up by taking a look at where I think this is going.
Next steps in Augmented Reality
One of the hot new trends in smartphones is the dual-lens camera, found in the LG G5, Huawei P9, Xioami RedMi Pro and rumored to be included in the upcoming iPhone7 plus. Adding a second image sensor to smartphone cameras will enable a lot of new features, including enhancing photo quality and a variety of editing capabilities. More importantly for AR, dual cameras will enable depth mapping and depth perception. This means that smartphones will be able to gauge the distance and size of objects in the real world, and AR apps could take advantage of that.
Google Tango video demonstrating tracking, AR overlay and other capabilities
In addition to fancy new hardware features like dual cameras, advances in deep learning and artificial intelligence (AI) will have a huge impact on AR especially in regards to the quality of the object modeling of the technology. As highlighted in Google I/O 2016, AI has stepped off of the science fiction shelf, and is entering the mass market in several different products.
Two exciting Google projects that will undoubtedly affect the AR world are Tango and Project Soli. Tango is designed to solve the problem of simultaneous localization and mapping (SLAM) by using sensors to map the immediate vicinity of the device. The first phone to include this technology is the just-released Lenovo Phab 2 Pro. This technology will enable detailed mapping of the ever-changing surroundings, and make the indoors (where GPS is unavailable) as navigable as the outdoors. This has endless implications for AR apps like Pokémon Go, and others yet to come.
I believe that combining these upcoming three technologies – depth perception, SLAM and deep learning based object recognition into future devices would be key factors and could indeed enable a slew of advances and much higher quality AR capabilities. Of course, when talking about augmented and mixed reality, we should end by mentioning the extremely high profile Magic Leap project, which is bound to generate its first product in the coming months. This anticipated release comes with the hope to change the AR landscape and maybe even combine a few of the above technology advances in a low power untethered device. We can surely expect a lot from these new innovations and exciting technology to come up soon.
Find out more
To learn more about CEVA’s award-winning intelligent vision processor, CEVA-XM4, click here.
Find out how CEVA is bringing deep learning and AI to embedded systems – learn about the CEVA Deep Neural Network framework, CDNN, here.
To find out about our super-low-power Bluetooth platforms, including Bluetooth low energy (BLE) and Bluetooth dual mode (BTDM) IP solutions, click here.
Yair Siegel
Director of Product Marketing, Multimedia, CEVA
