Finding more accessible ways to connect

In 2020, global searches for "how to say i love you in sign language" reached an all-time high. While social distancing has been difficult for everyone, it’s disproportionately affected people who are deaf or hard of hearing. Wearing face masks and connecting over video calls has helped to keep us safe, but it’s made it harder to follow conversation through lipreading and more difficult to follow sign language that’s confined to a small screen. 

So it makes sense that we’re searching for ways to communicate better. And I’m proud to work at a company that’s building technology to help people do just that. Products like Live Transcribe provide real-time transcriptions of what is being said so people can follow conversations even while wearing masks, and Live Caption automatically captions videos and spoken audio on your device and browser so no one has to miss out on their favorite content. 

Behind all this technology are Googlers, like Tony Lee, who makes our films more accessible through captions and whose family uses captioning technology every day. The Googlers working on this technology are passionate about making the world more accessible for the deaf and hard of hearing communities — communities that some of them are also a part of. Here’s a glimpse at a few of the Googlers working on accessible features and the motivation behind their work.

Dimitri Kanevsky, Research Scientist for Live Transcribe, also on the Hearing Accessibility team

“I came to Google to work as a speech researcher to improve speech recognition algorithms. I completely lost hearing in early childhood and waited for so long to have this speech recognition application like Live Transcribe that I am using now everywhere every day. I was happy to be able to finally talk easily to my granddaughters.”

Live Transcribe is available for free on most Android devices in more than 80 languages. It transcribes conversations in real time, which can be helpful to people who are deaf or hard of hearing when speaking with a hearing person during face-to-face meetings. It can also be used to transcribe audio from videos and podcasts.

Ken Harrenstien, Software Engineer on YouTube captions

“For me and most of the Deaf/HoH in particular, captions are essential for accessible video content. I went to MIT. While there, I didn't go to classes because I didn't know ASL and couldn't lipread the lectures or anything. Just books. After we launched captions on YouTube, MIT began uploading many class lectures to YouTube, and I was able to find these videos and finally see how all those professors had been trying to make their subjects interesting. I can enjoy classes, lectures, talks, how-tos on literally any topic, personal stories, ASL performances, live streaming events… It's like having my own immense library of ‘just books,’ except with video. Without captions, we cannot achieve Google's original mission of making the world's information accessible and useful.”

In 2009, we launched YouTube Automatic Captions which use speech recognition to automatically add captions to videos. Since then, we’ve expanded to more than 10 commonly spoken languages and, to date, over 1 billion YouTube videos have been captioned.

Ricardo Garcia (Rago), Tech Lead on Sound Amplifier

“While most hearing assistive technologies focus on speech enhancement or intelligibility, we understood early on that the hearing experience goes beyond speech. Natural sounds — like birds chirping, a river stream or the buzz of a mosquito — and audible cues — like hi-hat cymbals or even the failing motor of a fridge — are well beyond the speech frequency range. I truly believe that by creating these audio algorithms, everybody benefits — not only the deaf and hard of hearing communities.”

Sound Amplifier is an Android application that when used with headphones allows you to filter, augment and amplify the sounds in your environment. It increases quiet sounds, while not over boosting loud sounds and gives the user control over noise reduction and sound enhancement settings.

Sean Kinzler, Software Engineer on Meet Captions

“I joined Google to work on features that are not only challenging, but also have a significant impact on people's lives. Since the start of COVID-19, virtual meetings have become a critical part of maintaining communication lines in the workplace. Making it possible to caption these meetings helps the Deaf and Hard of Hearing community more freely join in and have an impact in virtual meetings.”

Google Meet uses speech-to-text technology to provide live captions in meetings, which helps participants who may be deaf or hard of hearing follow along and stay engaged. The feature is available in several languages, including French, German, Portuguese, Spanish and English.

I am hard of hearing myself, and throughout my life there have been many times I have felt left out of conversations, or wanted to engage in the media content I was watching, but couldn't. I just wanted to be included and thought about. Working on accessible features has been my life’s work, and I am thankful that in my role at Google I’ve been able to help make sure all disabled people are included in the work we do, and the stories we tell. Across our products and stories, we aim to create a truly accessible world.