Google’s Project Tango & The Mobile Device of the Future

Google’s Advanced Technology and Projects (ATAP) group is not your ordinary corporate outfit. The tagline for their Google+ page, after all, is “We like epic shit.” I and several other members of the 3Pillar team had the opportunity to get hands-on experience with one of ATAP’s most promising creations at a recent Google Project Tango hackathon in Timisoara, Romania.

In this blog post we will cover what Project Tango is, the hackathon experience and the applications we worked on during it, potential uses for innovative devices like Project Tango, building apps for it, and the impact devices like Project Tango will have on mobile devices of the future.

What is Project Tango?

Project Tango is a mobile phone prototype that has the ability to interpret 3-dimensional space and motion. Google and a number of research partners spread across 9 countries have spent the last 10 years working to create Project Tango. The current 5” Android prototype comes equipped with high-end hardware and software that helps it track the complete 3-D motion of the device. The device captures over a quarter million measurements per second and turns the information into a single 3-D model of the environment around the user. The video embedded below that tells the story of Project Tango has been viewed more than 2.5 million times since being uploaded in February.

The Hackathon Experience

You know a hackathon is going to be special when you are asked to sign an NDA before you can officially participate. It was a great opportunity for us to get the chance to be among the select few who could work with devices that are now in the prototype phase but one day soon may be an indispensable part of our lives.

One of the biggest challenges such a project brings is that you have little to no documentation to work with. The complexity compounds when you are working with a “tricky device” that has state-of-the-art hardware and software that is not yet included in even the most conventional devices.

The fact that you are in an environment with like-minded professionals is a definite plus. You end up brainstorming in the kind of creative environment that comes with such an event. The 3Pillar team was a robust mix of people and disciplines, and that also helped. From yours truly taking care of the UX and engineering, to Daniel Markovits (Business Analyst), Nicolae Titus (Software Engineer), and Ciprian Mester (Technical Manager). They were also well supported by Mihai Dragoi, Software Developer and C programming expert who works at Saguaro Print in Romania.

Our Applications

Because we were basically diving into uncharted waters with this project, we decided to start working on three separate projects during the hackathon. The reasoning behind this was to cover all the new features this phone brings to the table, and to see which of the projects was more feasible based on what the device can do, as we didn’t know exactly the device’s capabilities until the last moment.

The projects that we worked on were:

1. An application for visually impaired people, which would guide them to avoid obstacles in a room by providing audio feedback to detect the location of closest object to you. The concept here is similar to how bats use echolocation to detect potential obstacles in their flight path. The problems we ran into here were more on the audio side than on the actual device itself. This project was dropped relatively quickly because of those problems.
2. A 3-D scanner that will allow you to scan a large real-life object using only the device and show you live feedback on the device of the scan progress. Here we faced a limitation in our knowledge of 3-D optimization in order for the 3-D model to be rendered on mobile devices.
3. A game that enables you to kick a virtual ball around the room you are in and collide with the objects in the room.

From these, we decided to go with the last one. The hackathon was only 48 hours and we wanted to end with something close to a finished product. While we didn’t manage to finish it during the hackathon, in the week that followed we had a chance to still continue to work on it and we actually managed to bring it to a usable state.

Potential Uses for Devices like Project Tango

The real measure of success for any new product is how it will add value or functionality beyond what already exists. Some of the areas that Google envisions Project Tango-like devices providing utility are:

• Retail – mapping inventory in stores so that people know where to find exactly what they are looking for
• Use by the Visually Impaired – to navigate their way around the physical world
• Augmented Reality – taking games and advertising out of your screen and placing them in the world around you
• Extending the current photo and video applications with depth sensing capabilities

Building Apps for Project Tango

For the time being, if you’re a developer you’re in the dark on how to build apps for Project Tango. Everything is done behind closed doors for this project, more or less, at this point. That is expected to change soon, however. The Wall Street Journal, Verge, and a number of other tech media outlets have run stories recently on the soon-to-be debuted 7” tablet version of Project Tango.

According to those sources, Google will be distributing 4000 tablet prototypes to developers in the near future, perhaps even in advance of their annual I|O Conference in late June. While it’s too early for companies like retail stores to build apps for devices like Project Tango, it’s never too early to start imagining how they can help a customer with a portable Kinect in their pocket, more or less. Imagining is where they’ll have to stop – for now – as all the APIs and most of the capabilities of the device are being kept secret.

What Project Tango Means For the Future of Mobile

While the smartphones on the market that we all know and love can do amazing things, we have only begun to scratch the surface of what will truly be possible with mobile devices.

It was about time to improve the functionality and capabilities of a smartphone’s camera beyond just the megapixel realm, for example. This device gives smartphones the ability to sense the distance to the pixels captured beside the color, for example.

Project Tango is also another great example of the continuous miniaturization path that technology tends to follow lately. This phone is basically a mobile Kinect device, as seen in iFixIt’s teardown of the device, which is embedded below.