A picture is worth a thousand calories. Boston startup Lose It! just released a new function of its food-tracking app called Snap It. With Snap It, users take a picture of the food they’re about to devour and the app uses image recognition technology to identify it, logging all of the nutritional information.
That’s the claim anyway. I had the honor of accessing Snap It before the beta was released to the public. Now, I don’t know what to believe: Does Snap It work, or do I just eat freaky food?
To set the scene, here's a quick video showing how Snap It works, with my black bean pasta with roma tomatoes, artichoke hearts and olives serving as the star of the show. You're witnessing one of the more successful Snap It experiences I had over the past week and a half.
According to Lose It!:
Research universities and prominent companies have for several years pursued initiatives to identify foods in photos using deep learning. Until now, FoodDist, the food image analysis model developed by researchers from Cornell and UCLA, has been lauded as state-of- the-art in comparison to all published benchmarks. The framework behind Snap It uses some of today’s most advanced image recognition technology, to beat the benchmarks, and deliver the best consumer food tracking experience in the world.
Yet, Snap It struggled to identify most of my meals.
For example, I came home to my Back Bay micro-apartment Monday evening and pulled out my copy of Mastering the Art of French Cooking to make onion soup from scratch. Before slurping away, I used the new app to snap a picture of the steaming bowl before me, and it made its best guesses: a lemon, sauce or applesauce. Julia rolled over in her grave.
The following day was no better. It suggested my yogurt was ice cream of some sort or, my personal favorite, mayonnaise. Because true champions eat mayonnaise for breakfast. How about that sandwich I brought for lunch? Surely it must be either a chicken leg, oysters or a dumpling, the app told me. None of the above, but I wish.
Safe to say 95 percent of the time, I ended up giving up on the pictures and manually searching within Lose It! for the food I was really eating. Or I thought to myself, “Screw this. I’m hungry,” and exited the app altogether before logging a single morsel.
At the Snap It launch party, I wanted an explanation.
Charles Teague, founder and CEO of Lose It!, gave me one. Teague likened the beta version of Snap It to when Facebook first released its face recognition feature for tagging: It was a bit off the mark at first, but only got more accurate with time. And data.
He told me, “We want to get as many people as possible and have them take as many pictures as possible with this beta. That’s how deep learning works. You see the broader vision of what it will do, but you know it needs to learn, to analyze more information and adjust itself before that broader vision becomes real.”
That's a valid point. Elyse Winer, VP of marketing at Lose It!, further explained that the more people use Snap It, the more the food database will grow. Ergo, the app will be better at identifying meals and determining portion size, and food suggestions will become more personalized and nuanced - all with minimal manual intervention. So Snap It will eventually know the difference between hummus and baba ganoush, and surmise that if you're eating sushi, it's probably made with brown rice because that's how you like it.
In the meantime, Snap It serves more purposes than engaging in a culinary guessing game. Winer said, "It's about mindfulness. When you have to stop and take a picture before you eat, you have to acknowledge what you're about to eat."
She also pointed out that the feature already fits into consumer behavior these days, what with the rise of food-flavored Instagram posts. So while you're grabbing a photo of that pho for your Insta-followers, you could just as easily log a meal, all of its caloric value and nutritional information.
I acknowledge what they're saying about deep learning is true. Same about the potential value Snap It poses. Hell, if I really wanted to lose more than three pounds and had the patience to log every, single speck of food that went into my mouth, I'd constantly use Snap It. But today, I'm looking forward to enjoying my morning yogurt - or maybe mayonnaise - with my phone sitting securely in my bag.
Videos and images via Olivia Vanni.
