What we are thinking about

Don’t you hate it when you are almost done cooking dinner, then realize you are missing a key ingredient? Those days are over, thanks to Chefling!

Guest Post in collaboration with Jeff Xie (VP of Engineering) & Arpan G. (Software Engineer)

Chefling is the perfect example of how voice-first leads to intelligent and delightful experiences. The kitchen assistant brings intelligence to your food pantry. With the skill, when you ask for what to cook tonight, Chefling knows what you have in stock and recommends a personalized recipe based on what you have.

As you run out of items, you can add things to your shopping list. Soon, you’ll be able to re-order these items directly from Chefling.

“Our ultimate goal is to make Chefling the operating system of choice for smart fridges and provide every family with a truly intelligent kitchen. Future plans include incorporating advanced A.I. elements, voice commands, and image recognition technology so that Chefling will become a virtual assistant that people can engage with in a meaningful conversation.”

“40% of US Food is wasted”

The future of voice in the kitchen is boundless. With the US Food Waste up to 40%, it is important to know which food you should use sooner and what delicious meals you can make with it — Chefling can help with that. Chefling uses VoiceLabs, and by analyzing Voice Pathing the company uncovered that the most used commands are adding inventory to the pantry and adding things to the shopping list.

Two key drivers of voice in the kitchen are:

  1. To get things done quickly
  2. To be hands-free.

There is a high probability your hands are dirty or busy cooking and voice is a more natural consumer experience to assist you. You’re able to get things done quickly by using your voice to set the timer, change the temperature of the oven, add things to your shopping list, check on your sous vide and ask for a recipe to cook.

“With Speech Finder, we were able to get a better understanding of how people were engaging with recipe searches. We had to think about the what voice apps are best for: the voice recipe search to match food in your pantry is great, but you may not want to cook it if you can’t see what the dish looks like. Making sure our voice app covers the key use cases was a good exercise to think about.”

The Kitchen of the Future – where’s Jarvis from Ironman?

How does a voice assistant help put a final touch to a dish – does it recommend an extra touch of truffle oil; does it play a song that matches the cuisine; or does it coach you on how to do 10 things at once as you are making dinner for a big family. These are things that “Jarvis” should be able to do.

However, a fully connected kitchen is tough, it will require more IoT devices and sensors to know when things are fully ready and personal preferences. We are a long way from a fully automated kitchen, but with Chefling there is no doubt the kitchen is more intelligent.

In the immediate future, Chefling is thinking about engagement across devices with visual and voice — the kitchen needs a screen. This will lead to more people actively cooking using voice devices when making every meal. Video content and voice is coming soon, when you can talk with Android TV or Amazon Echo Show.

“One question we ask is should we think of voice as an extension of mobile, or its own application? I think adding a screen to voice devices will help us answer that question.”

Working with VoiceLabs

We asked Chefling for feedback on us:

With VoiceLabs, we are able to dive into how users are engaging with our intents that we built, which one’s they aren’t using as much, and create a hypothesis on how to educate a user or not spend any more engineering time on a particular intent.

We’re hoping at some point we’ll be able to get a better connection to users or user identity. If a user has an issue, it is hard to reproduce the bug without attributable user identity.  

It was very easy to integrate on both Alexa and Google Assistant — it took about 30 minutes. We really enjoyed the Speech Finder, to actually see the user output and what people are saying to Chefling.

After integrating, there was actionable feedback. From the product side, we were able to see the most used intents and prioritize our roadmap to enhance them — there is a lot of room for improvement and we’re working on it. In addition, we were able to see what items people were adding and we’re going to make enhancements based on this behavior.

Voice Development Wish List

We asked Chefling what would help them in future voice development:

#1 Community. In the voice development process, the community could be stronger. In web and mobile, there’s already community engagement, best practices, open source libraries, and frameworks. However, in voice development, there’s not too much out there. (Note: Chefling has since joined the VoiceLabs customer community to engage with other voice developers.)

#2 Better Education of Capabilities. For a relatively complicated skill, like ours, and how do you let people know that they can ask more questions? If they don’t know or don’t look at a tutorial, how do you train them? It’s a difficult paradigm for all skills & apps on Amazon Alexa and Google Assistant. Discovering voice commands — what’s a good tool to educate the user? Is it a developer problem or a platform problem?

“Our dream for the future would be more similar to WestWorld. You don’t need to know what you can or can’t ask. Right now you have to remember what questions you can ask, which on its own is a mental barrier.”

#3 Discovery.  If we had more users, we’d be able to do a lot more. At each stage, we’re going to be user-centric and listen to the consumers to see what we should build. Discovery on both platforms is severely lacking.

When we do reach a significant user threshold, we will have a very sophisticated record of people’s kitchen behavior (i.e. how often they purchase some items, what brand might attract them, etc). Therefore, we can make our inventory management system more intelligent; including, reminding users to buy certain items or providing valuable purchase suggestions to users. It will get closer and closer to Jarvis.