Google is Trying to Count Calories in Food Porn

What hidden information do your foodie Instagrams hold? A lot, according to a research scientist at Google.

Recently at an innovation summit in Boston, Google's Kevin Murphy unveiled a program project called Im2Calories. Using a series of algorithms, the program is designed to analyze any image of food and tabulate the number of calories in the image, Eric Sofge reports for Popular Science. The program is spawned from branch of computer science and artificial intelligence research called "deep learning" where complex algorithms try to model abstract things (in this case calories) in a data set (in this case food images) basically using high-level math.

Im2Calories is designed to recognize the type of food on a plate from pixel depth and patterns of what different food items look like in existing images. It can calculate the size of the food item compared of the rest of the plate, and draw from the wealth of caloric data that's already out there on how many calories go into different foods, adding the calories from any visible condiments. For example, at the summit, Murphy computed the amount of calories in an image of someone's breakfast of two eggs, two pancakes and two pieces of bacon. For the program to work, photos don't need to be high resolution. A regular, old Instagram shot should do.

The program is also built to learn as it goes, and get better at estimating calorie counts in a given image. For this reason, Murphy doesn't seem to be too worried about accuracy just yet. "Maybe we get the calories off by 20 percent. It doesn't matter. We're going to average over a week or a month or a year," he told Sofge. "And now we can start to potentially join information from multiple people and start to do population level statistics." As a result, over time, the app will ideally become more precise. Google seems to have a lot of faith in the prototype program: They've already filed a patent.

The larger context of the project seems to be aimed at fighting obesity as a public health concern in the United States. All this counting could make a person feel not so great about what they're eating, but Google's aim isn't to shame people struggling with their weight. The goal of the program is simply to make the process of calorie counting easier. Rather than plugging numbers into a spreadsheet or an app, estimates could be almost instantaneous.

Helen Thompson | | READ MORE

Helen Thompson writes about science and culture for Smithsonian. She's previously written for NPR, National Geographic News, Nature and others.