Why subtitles are suddenly everywhere and how they got there

People with hearing loss have a new ally in their efforts to navigate the world: subtitles that are not limited to their television screens and streaming services.

The COVID pandemic disrupted daily life for people everywhere, but many of those with hearing loss took the resulting insulation particularly hard. “When everyone wears a mask, they’re totally unintelligible to me,” said Pat Olken of Sharon, Massachusetts, whose hearing aids were insufficient. (A new cochlear implant has helped her a lot.)

So when her grandson’s bar mitzvah was streamed on Zoom early in the pandemic, well ahead of the service being offered captions, Olken turned to Otter, an app made to transcribe business meetings. Reading along with the ceremony’s speakers made the app “an amazing resource,” she said.

People with hearing loss, a group estimated to be approximately 40 million American adults, have long used technologies to help them navigate the hearing world, from Victorian-era ear trumpets to modern digital hearing aids and cochlear implants.

But today’s hearing aids can cost upwards of $5,000, often aren’t covered by insurance, and don’t work for everyone. The devices also don’t click audibly into focus, like glasses instantly correct vision. Instead, hearing aids and cochlear implants require the brain to interpret sound in a new way.

“The solutions out there are clearly not a one-size-fits-all model and don’t meet the needs of many people based on cost, access, and many different things,” said Frank Lin, director of the Cochlear Center. for Hearing and Public Health at Johns Hopkins University. That’s not just a communication problem; researchers have found correlations between untreated hearing loss and higher risks of dementia.

Cheaper over-the-counter hearing aids are on the way. But for now, only about 20% of those who could benefit from hearing aids use one.

Captions, on the other hand, are usually a lot easier to access. They have long been available on modern television sets and are increasingly popping up in video conferencing apps such as Zoom, streaming services such as Netflix, social media video on TikTok and YouTube, cinemas and live art venues.

In recent years, smartphone apps such as Otter; Live transcription from Google; ava; InnoCaption, for phone calls; and GalaPro, for live theater performances, have emerged. Some are aimed at people with hearing loss and use human reviewers to ensure that the captions are correct.

Others, such as Otter and Live Transcribe, instead rely on so-called automatic speech recognition, which uses artificial intelligence to learn and record speech. ASR has problems with accuracy and delays in transcribing the spoken word; built-in biases can also make transcriptions less accurate for the voices of women, people of color, and deaf peoplesaid Christian Volger, a professor at Gallaudet University who specializes in accessible technology.

Jargon and jargon can also be a stumbling block. But users and experts say that ASR is much improved.

While welcome, none of these solutions are perfect. New York’s Toni Iacolucci says her book club could be empty even if she used Otter to transcribe the conversation. The captions weren’t always accurate and didn’t identify individual speakers, which could make it difficult to keep up, she said.

“It kind of worked,” said Iacolucci, who lost her hearing nearly twenty years ago. When she got home, she would be so tired of trying to follow the conversation that she had to lie down. “It just takes so much energy.” She received a cochlear implant a year ago that has significantly improved her hearing, to the point where she can now have one-on-one conversations without closed captioning. They still help with group discussions, she said.

Otter said in a statement that it welcomes feedback from the deaf and hard of hearing, noting that it now offers a paid software assistant that can join virtual meetings and transcribe them automatically.

Transcription delay can pose other problems, including the concern that delays make interlocutors impatient. “Sometimes you say, ‘I’m sorry, I just have to look at my captions to hear it,'” says Richard Einhorn, a musician and composer in New York. “That doesn’t mean I’m not aware that it’s sometimes difficult for other people.”

Other problems crop up. When Chelle Wyatt of Salt Lake City went to her doctor’s office, the WiFi wasn’t strong enough for the transcription app to work. “It was gestures and writing things down and making sure I got a written report afterwards so I knew what was being said,” she said.

Movie theaters offer devices that amplify the sound, as well as glasses and individual screens that show captions that match the movie. But they are not always comfortable and sometimes not well maintained or simply do not work. Many people with to belong loss wants more movies to be subtitled on the big screen, just like you would in the comfort of your own home.

A new law that went into effect in New York City on May 15 requires: cinemas to offer on-screen captions for up to four showtimes per movie per week, including during the most popular movie times – Friday nights and weekends. Hawaii passed a state law in 2015 that required two screenings a week of each movie with on-screen captions. AMC, the major film chain, also says it shows some captioned movies in about a third of its US movie theaters.

Captions are now also more available for live performances. Several Broadway theaters are promoting a smartphone app that subtitles live performances; there are also portable individual devices that display captions. Theaters also have a few “open captioned” shows for all to see.

During the pandemic, the shift to online meetings and school meant video conferencing services became a survival tool, but closed captions came after a big push. Zoom didn’t add live transcription to its free service until October 2021, but the meeting host must enable it. Google Meet was faster in May 2020 with making captions available to everyone for free; Microsoft Teams, a workplace messaging app, did just that in June.

“We need captioning everywhere and we need people who are more sensitive,” Olken said. “The more I advocate, the more other people benefit from it.”

---

---

© 2022 The Associated Press. All rights reserved. This material may not be published, broadcast, rewritten or redistributed without permission.