Loud and clear: AI is improving Assistant conversations

To get things done with the Google Assistant, it needs to understand you – it has to both recognize the words you’re saying, and also know what you mean. It should adapt to your way of talking, not require you to say exactly the right words in the right order.

Understanding spoken language is difficult because it’s so contextual, and varies so much from person to person. And names can bring up other language hiccups — for instance, some names that are spelled the same are pronounced differently. It’s this kind of complexity that makes perfectly understanding the way we speak so difficult. This is something we’re working on with Assistant, and we have a few new improvements to share.

Teach Google to recognize unique names 

Names matter, and it’s frustrating when you’re trying to send a text or make a call and Google Assistant mispronounces or simply doesn’t recognize a contact. We want Assistant to accurately recognize and pronounce people’s names as often as possible, especially those that are less common.

Starting over the next few days, you can teach Google Assistant to enunciate and recognize names of your contacts the way you pronounce them. Assistant will listen to your pronunciation and remember it, without keeping a recording of your voice. This means Assistant will be able to better understand you when you say those names, and also be able to pronounce them correctly. The feature will be available in English and we hope to expand to more languages soon.

A good conversation is all about context

Assistant’s timers are a popular tool, and plenty of us set more than one of them at the same time. Maybe you’ve got a 10-minute timer for dinner going at the same time as another to remind the kids to start their homework in 20 minutes. You might fumble and stop mid sentence to correct how long the timer should be set for, or maybe you don’t use the exact same phrase to cancel it as you did to create it. Like in any conversation, context matters and Assistant needs to be flexible enough to understand what you're referring to when you ask for help.

To help with these kinds of conversational complexities, we fully rebuilt Assistant's NLU models so it can now more accurately understand context while also improving its "reference resolution" — meaning it knows exactly what you’re trying to do with a command.  This upgrade uses machine learning technology powered by state-of-the-art BERT, a technology we invented in 2018 and first brought to Search that makes it possible to process words in relation to all the other words in a sentence, rather than one-by-one in order. Because of these improvements, Assistant can now respond nearly 100 percent accurately to alarms and timer tasks. And over time, we’ll bring this capability to other use cases, so Assistant can learn to better understand you.

These updates are now available for alarms and timers on Google smart speakers in English in the U.S. and expanding to phones and smart displays soon.

More natural conversations

We also applied BERT to further improve the quality of your conversations. Google Assistant uses your previous interactions and understands what’s currently being displayed on your smartphone or smart display to respond to any follow-up questions, letting you have a more natural, back-and-forth conversation.

If you’re having a conversation with your Assistant about Miami and you want more information, it will know that when you say “show me the nicest beaches” you mean beaches in Miami. Assistant can also understand questions that are referring to what you’re looking at on your smartphone or tablet screen, like [who built the first one] or queries that look incomplete like [when] or [from its construction]. 

There's a lot of work to be done, and we look forward to continue advancing our conversational AI capabilities as we move toward more natural, fluid voice interactions that truly make everyday a little easier.