An App That Captures Emotions In Real Time

When you think of Intel, you probably think of processors and laptops—not Gumby and instant messages. A new app, Intel’s first direct-to-consumer offering, will turn that assumption on its head.

Pocket Avatars, which the company debuted last week, is an over-the-top messaging service that ports a user’s facial expressions onto a cartoon avatar, which in turn relays messages to friends and family.   

The app leverages your iOS or Android phone’s front-facing camera to capture a real-time image of your face. From there, facial-tracking algorithms monitor your expressions and superimpose them in real time onto your chosen avatar. As you speak, the app records your voice and syncs it up with the cartoon. Messages, which can run up to 15 seconds in length, zip to friends and family the same way a Facebook or WhatsApp message would. The company says the system is secure: it doesn't keep any of your data on file.

Intel partnered with several brands to build a catalog of move than 40 avatars, which users can download for free or buy for $0.99 each. Characters include Gumby, Lego minifigures and Grumpy Bear.

At a time when face-tracking is getting a lot of attention in both the consumer electronics and security industries—Microsoft's Kinect for Windows, for example, allows game developers to build apps that play off of a user's facial cues, and similar facial mapping tools such as the FBI's biometric system can be used to identify people—Pocket Avatars might be the most-accessible demonstration of its potential for the layman.

New Pocket Avatars App from Intel

Watch on

Developers didn’t initially set out to make a messaging app, says Rich Hannah, a director at Intel Labs, who led the team behind the project. “It never started as ‘let’s make an over-the-top messaging service,’” he recalls.  “The team was experimenting with video chat via avatar, webcams and a whole litany of things.”

While Hannah’s team was dabbling with facial tracking on various computer platforms, they ended up gravitating towards mobile devices. That’s when Intel’s product development and business teams got interested. And with good reason: Over-the-top messaging is a hot market. In 2013, users sent more than 10.3 trillion such messages, a number that’s expected to double in 2014. The worldwide text-message count in 2013 was about 6.5 trillion.

From there, Pocket Avatars was on the fast track. “Technology can sit in the labs for a long time,” he says, “but it moved through product development quite quickly.”

Currently, the algorithm that Hannah’s team developed can register subtle facial movements—an eyebrow raise, for instance—track head movement in all four directions, and zoom in and our on the character based on your distance from the camera. The avatar will follow every tick in real time, as if you were looking into a magic cartoon mirror. 

The avatar can currently mimic a slew of expressions (Intel isn’t releasing a specific number) and more will come as developers push out app updates.

“We have many micro-expressions already built in the model file,” Hannah explains, “And we’ll be able to unlock them and evolve the expressions quickly.”

Intel isn’t revealing Pocket Avatars’s secret sauce, but most facial-tracking software—whether it’s a webcam app or high-end motion capture for a movie—works in a similar fashion. The software homes in on key points on a person’s face (say, the corners of the mouth and the points of eyebrows) to determine what expression the person is making in that moment. The system then maps that emotion onto a character.

For the time being, Hannah says his team is focused on improving the in-app experience for Avatars (a forthcoming update will help improve the app’s performance in less-than-ideal lighting conditions, for example) rather than looking for other ways to use the tracking technology. For instance, in the future, a webcam might be able to read your facial expression and add an accurate representation of how you're feeling to a Facebook status.

But, as we continue to rely more and more on virtual communications the ability to share emotions in real time will become increasingly important.

“There are lots of potential applications for facial-tracking capability,” he admits, “but we’re not revealing how we’re going to apply this technology [in the future] right now.”

Corinne Iozzio | | READ MORE

Corinne Iozzio is a New York–based technology writer and editor. When she’s not fiddling with LEGOs or Nerf blasters, she covers gadgets and emerging tech for various publications, including Popular Science and Scientific American.