Inside Apple: Sonic Accessibility

Summary

Twenty Thousand Hertz – Episode Summary: Inside Apple: Sonic Accessibility

Release Date: April 2, 2025
Host: Dallas Taylor

---

Introduction

In this episode of Twenty Thousand Hertz, host Dallas Taylor delves into Apple's groundbreaking advancements in accessibility, particularly focusing on sonic accessibility. Through an insightful conversation with key Apple figures—Sarah Herlinger, Deidre Kaldbeck, Ron Huang, and Eric Tresky—the episode uncovers how Apple integrates accessibility into its core values, transforming lives through innovative sound-related technologies.

Apple's Commitment to Accessibility

Dallas Taylor opens the discussion by highlighting Apple's long-standing dedication to accessibility. He emphasizes that many of today's ubiquitous audio technologies, such as voice commands and text-to-speech, originated from efforts to assist individuals with disabilities.

Dallas Taylor [02:00]:
"If you want to know where the world of sound is headed, accessibility is a great place to look."

Sarah Herlinger, Senior Director of Global Accessibility Policy and Initiatives at Apple, elaborates on this commitment. She underscores Apple's philosophy of "nothing about us without us," ensuring that accessibility features are developed in collaboration with the disability community.

Sarah Herlinger [03:20]:
"Accessibility is something that's incredibly important to us. It is a part of the process in everything that we build."

Historical Context of Apple's Accessibility Features

Apple's journey in accessibility began in the mid-1980s, shortly after the introduction of the Macintosh computer. The first Office of Disability was established in 1985, setting the stage for decades of innovation.

Sarah Herlinger [04:02]:
"It's also not something that's new to us. Our first Office of Disability actually started in 1985."

One of the earliest features, MacIntalk, was a text-to-speech engine that paved the way for future developments like VoiceOver. By 2007, Apple introduced VoiceOver with a natural-sounding voice named Alex, which significantly enhanced the user experience for visually impaired individuals.

Tim Cook [05:03]:
"The 9000 series is the most reliable computer ever made."

VoiceOver and Universal Access

VoiceOver marked a significant milestone by integrating screen reading capabilities directly into Apple products, eliminating the need for expensive third-party software. This feature was part of the broader Universal Access initiative, symbolized by a blue Vitruvian man named Vito.

Sarah Herlinger [07:11]:
"We designed it with over 150 different breath sounds to make it sound more natural."

The introduction of Universal Access democratized accessibility, making it a standard rather than an add-on feature. Tim Cook highlighted Apple's unwavering commitment to accessibility as a core value, transcending financial considerations.

Tim Cook [08:19]:
"If you think back to how Apple was founded and is still the case today, we make tools for people to do incredible things and change the world with them."

Advancements in iPhone Accessibility

With the release of the iPhone 3GS in 2009, Apple revolutionized mobile accessibility. Features like VoiceOver and early versions of voice control transformed how visually impaired users interacted with smartphones.

Sarah Herlinger [09:18]:
"When we designed it, we actually had to rethink all of the ways that one interacts with the device to make it a safe environment for someone in the blind community."

The subsequent introduction of Siri in 2011 further expanded voice command capabilities, making smartphones more accessible and enhancing usability for individuals with limited mobility or vision.

Tim Cook [11:24]:
"For people with limited mobility or vision, it was a game changer."

Integration of Accessibility in Apple Watch and Health Features

Apple extended its accessibility efforts to the Apple Watch in 2015, blending accessibility with health monitoring. The Noise app monitors ambient decibel levels, alerting users to potentially harmful sound exposure.

Deidre Kaldbeck [19:10]:
"We build our health features with the objective of making an impact on people's lives."

The Apple Watch's features have had unexpected positive outcomes, such as assisting individuals with autism in managing their vocal levels.

Deidre Kaldbeck [19:22]:
"The noise app was helping give him insight, real time, with how loud he was speaking."

Innovations in Hearing Health

Addressing the widespread issue of undiagnosed hearing loss, Apple introduced several cutting-edge features:

1. Live Listen: Utilizes the iPhone's microphone to transmit audio directly to hearing aids, enhancing conversations in noisy environments.

   Sarah Herlinger [21:07]:
   "Live Listen uses the iPhone's microphone as a directional mic and sends that audio directly into your hearing aids."

2. Sound Recognition: Leverages the iPhone and Apple Watch to identify environmental sounds like doorbells, alarms, and crying babies, providing visual alerts.

   Sarah Herlinger [23:51]:
   "The iPhone and the watch are able to listen for environmental sounds around you and present a visual alert."

3. Hearing Tests and Assistance: A clinically validated hearing test integrated into the iPhone and AirPods Pro allows users to create personalized hearing profiles, effectively turning AirPods into hearing aids.

   Deidre Kaldbeck [32:30]:
   "This feature brings together engineers, clinicians, audiologists, designers to build what is really the first of its kind hearing aid."

These advancements aim to reduce social isolation, improve cognitive health, and lower the stigma associated with using hearing aids by integrating assistance into everyday devices like AirPods.

Sarah Herlinger [35:22]:
"We're always just trying to make sure that we have features that work for everyone."

Adaptive Audio and Enhanced AirPods Features

Apple's Adaptive Audio technology in AirPods dynamically adjusts between active noise cancellation and transparency modes based on the user's environment, creating a personalized listening experience.

Ron Huang [26:10]:
"Adaptive audio dynamically blends active noise cancellation in transparency modes based on the environment you're in."

Additional features like Conversation Boost and Conversation Awareness enhance interactions by detecting when the user begins speaking and amplifying the conversation partner's voice while minimizing background noise.

Eric Tresky [29:58]:
"These new algorithms go a long way towards improving that."

Adaptive Audio not only improves listening quality but also fosters better social interactions by intelligently managing audio dynamics in real-time.

Conclusion: A Future Shaped by Accessibility

Apple's relentless focus on accessibility has positioned it at the forefront of sonic innovation, making devices more inclusive and enhancing the quality of life for millions. By integrating advanced technologies like AI and augmented reality, Apple continues to push the boundaries of what's possible in hearing health and accessibility.

Tim Cook [35:23]:
"Democratizing things so everyone can create whatever they would like to create or solve whatever problem they would like to solve, that's what we're about."

Deidre Kaldbeck [34:33]:
"These features can really make an impact on people's lives in ways they might not have imagined."

Through empathy-driven design and collaboration with the disability community, Apple ensures that accessibility remains a cornerstone of its innovation, paving the way for a more inclusive future in sound technology.

---

Notable Quotes with Timestamps:

• Dallas Taylor [02:00]:
  "If you want to know where the world of sound is headed, accessibility is a great place to look."

• Sarah Herlinger [03:20]:
  "Accessibility is something that's incredibly important to us. It is a part of the process in everything that we build."

• Tim Cook [08:19]:
  "If you think back to how Apple was founded and is still the case today, we make tools for people to do incredible things and change the world with them."

• Sarah Herlinger [09:18]:
  "When we designed it, we actually had to rethink all of the ways that one interacts with the device to make it a safe environment for someone in the blind community."

• Deidre Kaldbeck [19:22]:
  "The noise app was helping give him insight, real time, with how loud he was speaking."

• Ron Huang [26:10]:
  "Adaptive audio dynamically blends active noise cancellation in transparency modes based on the environment you're in."

• Tim Cook [35:23]:
  "Democratizing things so everyone can create whatever they would like to create or solve whatever problem they would like to solve, that's what we're about."

---

Credits:

• Produced by: Nicholas Harder and Casey Emerling
• Sound Design and Mixing: Jesus Certeaga and Brandon Pratt
• Guests: Sarah Herlinger, Deidre Kaldbeck, Ron Huang, and Eric Tresky
• Written and Produced by: Nicholas Harder and Casey Emerling
• Additional Assistance: Grace East

Listen to the full episode on Apple Podcasts, Spotify, or YouTube.

Transcript

[Dallas Taylor]

Hey listeners, this message is for all of the audio storytellers out there now. That could mean aspiring or professional podcasters, radio journalists, sound designers, voice artists, YouTubers and beyond. If you have a great sound related story to tell, then I want to hear it. For the first time ever, we're holding an official short story competition called Sound Off. This is your chance to submit a fully produced two to five minute story that will play for over 100,000 people on this podcast. It could include sound design, music, interviews, whatever you'd like. It just has to relate to sound in some way and be appropriate for all ages. If you've already made something that fits that criteria, you're welcome to submit it. We'll count down the winning stories in a special episode this summer the top creator will get on a private feedback session with our team, plus $100 in credit for our online shop. Submissions close on May 7, so get started now. For full details on what we're looking for and how to submit, visit 20k.org soundoff there's also a link in the show notes, that's20k.org soundoff thanks.

[Unknown]

Hear that Spring is here and the Home Depot.

Has great prices on grills to make this season yours.

So if you're working on improving your.

Hosting skills, you're going to want the next grill four burner gas grill for $229.

And of course, pair it with the next grill eight piece grill tool set.

Now get outside and show off those new skill Shop a wide selection of.

Grills under $300 at the Home Depot. You're listening to 20,000 hertz. I'm Dallas Taylor. I've always been fascinated with accessibility when it comes to sound because so many incredible innovations in the world of audio began as accessibility efforts. Take for instance, voice commands and text to speech. They were originally designed for people with visual or motor impairments, but today we take for granted that we can talk to our devices and they can talk back to us. Closed captions were created for people with hearing impairments. Yet regardless of your hearing, many of us use captions all the time. Even audiobooks were originally created by blindness advocacy groups way back in the 1930s, and now audiobooks are a mainstream, multi billion dollar industry. The point is, if you want to know where the world of sound is headed, accessibility is a great place to look. I firmly believe that many of the greatest future achievements in sound will come from accessibility efforts. Now.

[Sarah Herlinger]

Accessibility is something that's incredibly important to us. It is a part of the process in everything that we build.

[Unknown]

That's Sarah Herlinger.

[Sarah Herlinger]

I'm the Senior Director of Global Accessibility Policy and Initiatives at Apple.

[Unknown]

I met with Sarah at Apple park in Cupertino, California.

[Sarah Herlinger]

My team's job is to ensure that every way that Apple presents itself to the world, whether that be through our products, our services, our stores, our workplace events, you name it, that we are living our core value of accessibility as a basic human right.

[Unknown]

To embody this idea of accessibility as a basic human right, Apple depends on.

[Dallas Taylor]

The people who use these features in.

[Unknown]

Their day to day lives.

[Sarah Herlinger]

It starts with adherence to the disability community mantra of nothing about us. Without us, you don't build for a community, you build with them. And the first step of that for us is the hiring of people with lived experience on our teams to help drive the development of our different types of accessibility features. We look at accessibility as kind of falling into five main pillars, which is vision, hearing, physical, motor, cognitive, and speech. And we build features to support each one of those areas.

[Unknown]

For decades now, Apple has been continually refining its approach to those five pillars.

[Sarah Herlinger]

It's also not something that's new to us. Our first Office of Disability actually started in 1985.

[Unknown]

That was just one year after Steve Jobs first introduced the Macintosh computer. And during that announcement, it was actually an accessibility feature that stole the show.

But today, for the first time ever, I'd like to let Macintosh speak for itself.

Hello, I am Macintosh. It sure is great to get out of that bag.

[Dallas Taylor]

It was called macintalk and it was an early text to speech engine, otherwise known as tts. These enable programs to read text aloud. They're helpful for the visually impaired and people with learning or cognitive disabilities, basically anyone who might have trouble reading.

[Unknown]

So it is with considerable pride that.

[Dallas Taylor]

I introduce a man who's been like a father to me, Steve Jobs.

[Unknown]

Macintosh made the Macintosh look irresistibly cool, like something out of 2001 A Space Odyssey.

[Tim Cook]

The 9000 series is the most reliable computer ever made.

[Unknown]

Starting in 1987, many of the Mac's.

[Dallas Taylor]

Accessibility features were bundled under a label called EasyAccess. This included things like sticky keys which made it easier to use keyboard shortcuts and mouse keys which let you control your cursor with your keyboard. But maybe the most important was a third party program program made by a company called Berkeley Systems. It was a screen reader called Outspoken.

[Unknown]

Now, screen readers don't just read text. Instead, they take in everything on screen and turn all of that visual information into described audio.

[Dallas Taylor]

Here's an example of someone using a.

[Unknown]

Screen reader on a school's website.

[Dallas Taylor]

Navigation region heading level 2 link services for children. List of 4 items as they move around the screen with their keyboard arrows.

[Unknown]

The screen reader explains whatever element is highlighted. In this case, a set of links.

[Dallas Taylor]

Lingq early years Visited Lingq Primary school years Lingq High school and beyond.

[Unknown]

People who use screen readers often get used to listening at incredible speeds, some up to around 1,000 words per minute. Here's a demo on YouTube like this.

[Dallas Taylor]

I've been told it's a little fast, so 40%.

[Unknown]

How's that? Today, screen readers are really common, but up until the mid aughts the good ones were all super expensive.

[Dallas Taylor]

The leading screen reader for Microsoft computers.

[Unknown]

Was called Jaws, which could cost over $1,000.

[Dallas Taylor]

Ethernet explorer, space Editor, Region Search engine, hypertext. But in 2005, Apple changed the game by rolling out Voiceover, their first built in screen reader. Isn't it nice to have a computer.

[Unknown]

That will talk to you? It took a couple of years to work out the kinks, but by 2007 Apple had developed a serious competitor to Jaws. That same year, Voiceover got a new, more natural sounding TTS voice named Alex. In particular, people noticed how he breathed.

[Tim Cook]

This is Alex.

[Unknown]

I'm programmed with over 150 different breath sounds.

[Dallas Taylor]

You can still find me in the system settings. Voiceover was a new addition to a broader set of accessibility features that were now called Universal Access. To represent universal access, Apple designed a symbol of a blue Vitruvian man. Basically a little blue person with their arms and legs outstretched. At Apple they call him Vito. And now it's come to symbolize accessibility all around the world.

[Unknown]

Universal Access marked the beginning of a new era. Accessibility would no longer be led by third party companies who sold their software as expensive upgrades. Instead, Apple itself would take the lead and bake these features right into their products. Now, part of it may have been a business decision because the more accessible a device is, the more marketable it is to schools, libraries and other public institutions. But regardless of the financials, people like Steve Jobs and Tim Cook just thought it was the right thing to do. Here's Tim Cook with interviewer James Rath.

[Tim Cook]

If you think back to how Apple was founded and is still the case today, we make tools for people to do incredible things and change the world with them. And that's everybody. I've never ever in the 20 years of being at Apple ever looked at a what's our return on investment here? It wouldn't be Apple without doing this. I mean, it's a Part of our values that we will not compromise on.

[Dallas Taylor]

Universal access was a huge step forward for accessibility on computers. And yet in the mid aughts, most cell phones still weren't very accessible. Many people with disabilities were limited to the most basic functions of a phone.

[Unknown]

Making calls and sometimes texting. And like with jaws on windows, these often required expensive add ons. But all of that changed in 2009 with the iPhone 3GS.

[Sarah Herlinger]

When we designed it, we actually had to rethink all of the ways that one interacts with the device to make it a safe environment for someone in the blind community so that a single touch wouldn't make them do something they didn't intend to do.

[Unknown]

Up to that point, touch screens were much less friendly to the visually impaired than old fashioned tactile buttons. But once voiceover came to the iPhone, it transformed that experience just like it did with the Mac.

[Sarah Herlinger]

So for example, on an iPhone it will do everything from read your text to you to tell you how many bars of cell coverage you have or what time is it. You can just move your finger on top of all of the visuals on the screen, the icons, the words, and have it read back to you.

[Tim Cook]

Apps Setting Voice no Stocks Maps Weather Weather New York High 88 degrees Fahrenheit. Low 71 degrees Fahrenheit. Currently partly cloudy 82 degrees Fahrenheit.

[Dallas Taylor]

Along with voiceover, Apple also introduced an early version of what's now called voice control. If you have a disability that makes fine motor movements difficult, the ability to control your phone with your voice is vital. Now keep in mind, Siri still hadn't come out yet. But in this 2009 YouTube demo of voice control, we can hear something that sounds a lot like it.

[Unknown]

You just touch and hold down the home button for three seconds and it'll pop up.

[Tim Cook]

Help Using iPhone voice control you can tell iPhone2 call contacts play playlists Play songs by Collective soul Playing songs by Collective Soul.

[Unknown]

Then when Siri was officially introduced in 2011, it opened up even more of the iPhone to voice commands. Here's Apple's Scott Forstall showing off what Siri could do. Do I need a raincoat today?

[Tim Cook]

It sure looks like rain today.

[Unknown]

For many users, Siri was mostly a.

[Dallas Taylor]

Co futuristic new feature.

[Unknown]

But for people with limited mobility or vision, it was a game changer. You can send and receive text messages, you can create notes, you can search the web. You stick something in the oven, you're going to bake it and need to take it out in 30 minutes. Just take your phone and ask Siri to set a timer for 30 minutes and you're done.

[Sarah Herlinger]

We were the first company to make a consumer touchscreen accessible to someone in the blind community. It's not just voiceover, it's zoom, it's inverting colors, it's dynamic text. It's all these different things that are there so that whatever your personal unique need is, you can set up your device to work for you.

[Unknown]

And right away these features really caught people's attention. Around 2011, there was an outpouring of gratitude from the disabled community, especially users with visual impairments. For many of them, they jumped from barely having access to cell phones to having almost full access to the world's leading smartphone. At a 2011 event in Los Angeles, Stevie Wonder personally thanked Steve Jobs and.

[Tim Cook]

I want you to give a hand for someone who this company took the challenge in making his technology accessible to everyone, Steve Jobs. Because there's nothing on the iPhone or the iPad than you can do that I can't do.

[Dallas Taylor]

By this point, Apple had been releasing built in accessibility features for over 25 years. But this was only the beginning. Today, Apple is using AI, augmented reality, and all sorts of technology to make their devices more accessible and useful than ever before. These innovations are already changing how millions of people experience the sounds around them, and they have the potential to revolutionize the world of hearing health. That's coming up after the break. Ever since I started playing trumpet in my elementary school band, I've been in love with jazz music. I love the way it blends tradition with creativity, improvisation, and incredible skill. And the world of jazz goes so far beyond just the classics, stretching into funk, rock, R and B, hip hop, and much more. And that's what I love about yout'll Hear It, a fantastic podcast about the expansive world of jazz. The show is hosted by two musicians, Peter Martin and Adam Manus. They offer practical tips for musicians, review the latest jazz and jazz related albums, and break down what makes legendary recordings so timeless. In one recent episode, they explored Michael Jackson's iconic album off the Wall, including rare demos and interviews and even isolated music stems. In another episode, they broke down what makes music emotionally powerful, including specific chord progressions, rhythms, and melodies that evoke strong feelings in listeners. Whether you're a musician yourself or just a fan of great music, you'll find a lot to enjoy. You'll hear it Music explored Follow the show on Apple podcasts, Spotify or YouTube. Congratulations to Gillian McMaster for getting last episode's Mystery sound right. That's someone playing all of the different sizes of Boom Whackers. These are colorful plastic percussion tubes that produce a musical note when you hit them with your hand or against another surface. The smaller the size, the higher the note. Boom Whackers were first released in 1995 and are now made by a company called Rhythm Band Instruments. I've had Boom Whackers here at Taylor HQ ever since my first was born. Now here's this episode's Mystery Sound. If you know that sound, tell us at the web address mystery.2000. Anyone who guesses it right will be entered to win a super soft 20,000 Hz T shirt. Finally, a reminder that 20,000 Hz exists because of the work we do at my sound design company, Defacto Sound. So if you know somebody who works in video and their projects could use some Defacto sonic magic, then send them over to defactosound.com to hear what we do Shopping online can be an unpredictable experience. Sometimes all it takes is a few clicks and other times it's a convoluted pain in the neck. But if you want your customers to get the smoothest experience possible, then you need Shopify. Shopify is the selling secret behind companies like Allbirds, Cotopaxi and Alo Yoga. It's an all in one e commerce platform that makes checking out quick and seamless for every customer. That's thanks to Shopify's Shop Pay feature, which lets customers save their payment and shipping information across millions of Shopify stores. That leads to faster checkouts and up to 50% more conversions. In other words, less abandoned carts and more sales. Plus, Shopify lets you sell wherever your customers are on your website, in store and across social media. Upgrade your business and get the same checkout as companies like Aloe Yoga, Cotopaxi and Allbirds. Sign up for your $1 per month trial period at shopify.com 20k. Go to shopify.comtwo 0k to upgrade your selling today. That's shopify.com 20k. For many companies these days, there's a growing expense eating into their profits. And that's their cloud computing bill. Oftentimes you may get a deal to start, but it keeps increasing every year. But what if you could cut your cloud bill in half and improve performance at the same time? Well, if you act by May 31, Oracle Cloud Infrastructure, or OCI can help you do just that. OCI is the next generation cloud designed for every workload. It allows you to run any application, including any AI projects, faster and more securely.

[Unknown]

For less.

[Dallas Taylor]

In fact, Oracle has a special promotion where you can cut your cloud bill in half when you switch to OCI. The savings are real. On average, OCI costs 50% less for computing, 70% less for storage, and 80% less for networking. Join Modal, Skydance, Animation and today's innovative AI tech companies who upgraded to OCI and saved. This offer is only for new US Customers with a minimum financial commitment. See if you qualify for half [e-mail] 20k that's oracle.com 20k Apple's accessibility features go back to the 1980s, but they really accelerated in the aughts with universal Access in the iPhone3GS. Then when the Apple Watch came out in 2015, it included a suite of features that started to blur the line between accessibility and health. For instance, there's a noise app that constantly monitors the decibel levels around you. If it detects unsafe levels of sound, you'll get this alert.

[Deidre Kaldbeck]

We build our health features with the objective of making an impact on people's lives.

[Unknown]

That's Deidre Kaldbach.

[Deidre Kaldbeck]

I am the senior director of product marketing for Apple Watch and Health.

[Unknown]

Deidra says that the impacts of these features can sometimes be surprising, even to them.

[Deidre Kaldbeck]

When we first introduced the noise app, we heard from a father who said it's really changed how his autistic son experiences his life at school because he didn't know how loud he was speaking, and it sort of turned some people off that he communicated with. And so the noise app was helping give him insight, real time with how loud he was speaking and he could kind of bring the level of his voice down. And that was not the way the feature was designed. But those are the stories where we know, okay, we really want to invest more in hearing, we really want to invest more in all areas of health.

[Unknown]

One issue they've become especially focused on is hearing loss.

[Deidre Kaldbeck]

A lot of people don't know they have mild to moderate hearing loss. About a billion people around the world suffer from mild to moderate hearing loss, but 80% of those go undiagnosed.

[Sarah Herlinger]

The average person who needs hearing assistance generally doesn't get it for at times up to a decade after they should have.

[Unknown]

That's Sarah Herlinger again. Sarah says that even after someone seeks treatment for hearing loss, they can find themselves in a world that's not very friendly to the hearing impaired.

[Dallas Taylor]

For example, for a long time, hearing.

[Unknown]

Aids couldn't connect to a cell phone without installing a copper wire called a.

[Sarah Herlinger]

Telecoil, and the experience was not optimal. We were getting Emails from customers saying, you know, I know it's just telecoil, but I still can't find a way to make it really work well. And so I've stopped talking to my grandkids, you know, and people just got very insulated because they couldn't use the device as well.

[Unknown]

Isolation is a common symptom of hearing loss, especially when it goes untreated. After all, it's hard to be social when you can barely hear or understand a conversation. The same goes for watching a movie, going to a live event, or any social setting where you use your ears.

[Sarah Herlinger]

And what we looked at was how Bluetooth was really a great solution, but it wasn't something that was currently an option within the hearing aid world.

[Unknown]

So Sarah's team worked with the major hearing aid companies to bring Bluetooth to their products.

[Sarah Herlinger]

We actually wrote the first Bluetooth protocol for hearing and did it specifically for hearing aids, and built in a bunch of features like Live Listen that were specifically for hearing aid users and launched it in end of 2013, and it really revolutionized the hearing aid market.

[Unknown]

Live Listen uses the iPhone's microphone as a directional mic and sends that audio directly into your hearing aids.

[Dallas Taylor]

So let's say you go to a.

[Unknown]

Loud restaurant with a friend.

[Dallas Taylor]

What do you think you're going to order?

[Unknown]

Your friend can hold your iPhone up to their mouth and their words will be beamed into your Bluetooth hearing aid.

[Dallas Taylor]

What do you think you're going to order?

[Unknown]

And just like the noise app users have found creative new ways of using Live Listen. At one point, Sarah heard from someone whose mother had been losing her hearing, which made it hard to watch movies together like they used to.

[Sarah Herlinger]

It was always something he loved to do, and he went home for the holidays that year. And so what this guy did was he took her iPhone, turned on Live Listen and put it next to the speaker of the tv, and then they were able to start watching old Christmas movies together.

[Unknown]

I.

[Sarah Herlinger]

Dreaming of a white.

[Dallas Taylor]

Christmas.

[Deidre Kaldbeck]

I think that's been the most rewarding part of this process is to hear how much these products and features mean to people.

[Unknown]

Live Listen isn't just for hearing aids. It also works with AirPods and Beats. And like many accessibility features, it can.

[Dallas Taylor]

Be a useful life hack.

[Unknown]

Even if your hearing is perfect. For instance, you can turn up the TV for yourself without bothering anyone else or listen to it from the next room over.

[Dallas Taylor]

In recent years, Apple has been using AI and other innovations to push these features even further. For instance, there's Live captions, which generate live subtitles of any speech, whether it's a podcast playing on your phone or someone talking to you in person. There's also sound recognition, another potentially life changing feature for the hearing impaired.

[Sarah Herlinger]

The iPhone and the watch are able to listen for environmental sounds around you. Everything from a doorbell, a fire alarm, a dog barking, a baby crying, water running, and present you with a visual alert that says there is a sound behind you. It may be your water running.

[Unknown]

Now, you don't have to be deaf to find a use for sound recognition. For instance, you can use it to get alerted about your doorbell ringing when you have headphones on. But for people in the deaf community, it can be really impactful.

[Sarah Herlinger]

I remember when it first launched just seeing people who talked even specifically about that element of a baby crying and that incredibly human moment of having the realization your child is crying and to be able to go and pick them up and comfort them. And I think that one really more than maybe a dog barking or the doorbell ringing, it really just brings that human connection.

[Dallas Taylor]

Over.

[Unknown]

In the Magnifier app, there were similar features for the visually impaired called people detection and door detection. These allowed people to hold up their phone's camera and have it alert them about the presence of doors and other people.

[Tim Cook]

Open door 6ft away, turn handle or knob swing. Two doors detected, door 5ft away, turn handle or knob swing.

[Dallas Taylor]

More recently, Apple released something called scenes.

[Unknown]

Now, instead of just detecting doors and.

[Dallas Taylor]

People, the iPhone can describe many more details about your surroundings. It's like a screen reader for the real world.

[Tim Cook]

A group of people sitting in chairs in front of a desk with a laptop and a lamp. A person standing next to a glass door, a room with a couch, a table and other items.

[Unknown]

Door detection and scenes are great examples of augmented reality, a catch all term for when computer generated information gets overlaid on top of the real world. But there's another Apple device that's really pushing the envelope in this area and making augmented reality common for all users. And that's the AirPods.

It does so many more things in your life than just media.

That's Ron Huang, Apple's vice president of sensing and connectivity.

Our users tend to put it in and leave them in for a much longer time. And so that's why we build things like adaptive audio.

Adaptive audio came out in 2022, and it combines two older AirPods features into one. The first is active noise cancellation, which cancels out sound waves and as they enter your ear. The second is transparency mode, which is basically the opposite of noise cancellation. It's for when you want to be aware of your surroundings.

So adaptive audio dynamically blends active noise cancellation in transparency modes based on the environment you're in. So, for example, if you walk into a louder restaurant, we automatically ramp up the amount of noise cancellation we do to lessen the noise. And when you walk out of that restaurant, we then lessen that active noise cancellation so you get more of the transparency effect directly. Same thing goes if a truck drives by, right? Truck gets closer to you, active noise cancellation level raises. And when it drives away, we fall back down to transparency. It's not just a mode switch. It's literally a dynamic, gradual shift between the two modes. And the end effect is really special because what we hear from our customers over and over again is when they finally take the AirPods out, they have this OMG moment, which is like, I had no idea that the streets were so loud.

Features like adaptive Audio elevate the AirPods from Classic Earbuds into the world of sonic augmented reality. Essentially, it's changing our sensory input to make the outside world friendlier to our ears. When you're in a noisy environment, the world can feel like a bad mix where some of the instruments are super overpowering. But now we finally have some control over that mix. But just because you have AirPods in it doesn't mean you have to be closed off to social interaction.

[Dallas Taylor]

Two features that address this are conversation.

[Unknown]

Boost and conversation awareness. So let's say you're on the train for your morning commute and you're jamming out to some tunes, then you decide to ask someone a question.

And so what conversation awareness does automatically is that we real time detect when you start speaking. Microphones detect speech sounds, obviously, but there's potentially a lot of people around you also talking. And as we combine that with the accelerometer, so we know that it's from your jaw, it is you speaking, and therefore it is your intent to speak. And that's when we apply the ducking or the pausing of the audio to help you talk.

[Dallas Taylor]

Excuse me, do you know what the next step is?

[Unknown]

Once you start talking, the AirPods know that there's probably a reply coming that you want to hear.

So we beam from the mics to the conversation in front of you and actually use machine learning techniques to amplify the speech sound, but not the rest of the noise, so you can have a much better conversation.

[Dallas Taylor]

Should be Franklin. I'm actually getting off there. Got it.

[Unknown]

Thanks.

[Dallas Taylor]

The system even has a way of knowing when the conversation is over to do so they track your conversation partner using beam forming microphones and motion sensors.

[Unknown]

And when I'm done, as I walk away, we combine the fact that we detect you're walking away to also see that, oh, you're likely ending that conversation and therefore resuming the audio. Back to you.

Now for these features, all of the processing is happening inside the AirPods themselves. But of course, the processing power of an iPhone is much greater and in some cases it makes sense to utilize that. That's what Apple did with their voice isolation feature, which blocks ambient noise to make your voice more clear when you call or FaceTime someone.

[Eric Tresky]

We realized, especially with something like voice and all this machine learning capability is we have so much more compute power on the phone that we can take advantage of.

[Unknown]

That's Eric Tresky, who directs product marketing for AirPods.

[Eric Tresky]

So to remove that background noise, we actually now send a raw signal of your Voice directly from AirPods down to the phone. The phone does all the processing and then of course, that just goes out to your person on the other end.

[Unknown]

When I'm on a business call or facetiming with a loved one, I don't want my voice to sound like a noisy, garbled mess. And these new algorithms go a long way towards improving that.

[Eric Tresky]

And that's in windy conditions, that's in loud environments. So it's an incredible capability that we now have.

[Unknown]

Today, Apple is leaning into hearing health more than they ever have in the past.

[Deidre Kaldbeck]

So the three pillars with our new hearing health features are protection, awareness, and assistance.

[Unknown]

On the awareness side, there are those loud noise warnings which even apply to the things you're choosing to listen to.

[Deidre Kaldbeck]

We wanted to make sure they had some awareness around how loud they might be listening to their favorite music or their favorite podcast and giving them the ability to automatically reduce those loud sounds so that they're always listening to their media at a safe listening level.

[Unknown]

On the protection side, they've added automatic hearing protection across all three of the AirPods noise control modes. This can reduce the environmental sound hitting your ears by up to 30 decibels.

[Deidre Kaldbeck]

So maybe you are in a windy city or it's people who tend to be in situations where they don't even realize how loud their environment noise is, like a subway, or if they have a profession where there's a lot of loud sounds around construction, et cetera. Hearing protection will actually ensure that background noise is suppressed so that your hearing is protected over a long period of time.

[Eric Tresky]

We talk about really what AirPods can do in people's lives beyond just music. Listening, and we think about that exhaustively. So I really like the story of not only are we notifying you and able to protect your hearing from what you're listening to from a media perspective, but we're also now protecting your hearing from an environmental sound perspective. So you're sort of covered both ways.

[Unknown]

But for me, the most exciting part of this is what Apple is doing with the third pillar, hearing assistance. In a hearing study they conducted, we.

[Deidre Kaldbeck]

Learned that about 75% of the people diagnosed with hearing loss were not using any sort of assistance. Well, that seems like an area where we can really make an impact.

[Unknown]

So Apple developed a new clinically validated hearing test and built it right into the iPhone and AirPods Pro.

[Deidre Kaldbeck]

This feature brings together engineers, clinicians, audiologists, designers to build what is really the first of its kind hearing aid. So you can use your iPhone, you can take about a five minute test, and in the comfort of your own home, you will get a personalized hearing profile as a result. And this feature will seamlessly transform your AirPods Pro 2 into a hearing aid kind of adjust based on your personal specified needs so that you can hear the world around you much better.

[Unknown]

It's hard to overstate how big of a deal this is. Millions of people buy AirPods and statistically, many of them have hearing loss and don't even know it. But by taking this test and potentially using their AirPods as hearing aids, they can avoid some of the downstream effects of hearing loss, like social isolation and cognitive decline. In other words, it can give people who might never have gone to the doctor life changing information.

[Deidre Kaldbeck]

The thing that makes us really excited about this feature is that in the way that they can just use these AirPods that they rely on in their everyday life, they can now use to really help impact their lives in a way that I don't think they would have imagined. It also has the potential of reducing the stigma because you're seeing people wear AirPods and that's the same set of headphones that you use to listen to your favorite podcast and you can use to have that hearing assistance that you might need. I have a list of people that I plan to have, and if they're listening, they know who they are.

[Unknown]

For years now, I've been talking about how hearable technology was eventually going to combine headphones, earplugs, hearing aids, virtual assistants, and more into one earbud like device that we can theoretically leave in all day. This is the kind of technology that I'm most passionate about because it goes so far beyond just convenience or entertainment. It's the stuff that literally changes people's lives and helps people connect with each other through sound.

[Deidre Kaldbeck]

For me, when I started to work on Apple Watch and then soon after Health, to be able to hear some of these stories we've been sharing today, I just felt very fortunate that this was my actual job, this is my profession that I get to work with these brilliant people that come up with these features that anyone can use and anyone has the potential of having their lives changed.

[Unknown]

Now. Designing for accessibility comes with a lot of challenges, but when you approach those challenges with empathy and creativity, the result is often a better product for everyone.

[Sarah Herlinger]

We're all unique in the world and accessibility features may be life hacks to one person and they may be necessities to another, but we're always just trying to make sure that we we have features that work for everyone.

[Unknown]

Here's Tim Cook again.

[Tim Cook]

And so that basic thought of democratizing things so everyone can create whatever they would like to create or solve whatever problem they would like to solve, that's what we're about. That's why we're here for everything we build.

[Sarah Herlinger]

It's really just about giving people opportunity. I'd like to think that whoever finds the cure for cancer probably isn't going to look like what Hollywood has told us they should look like. And so if we can build a feature that unlocks someone's capability to express themselves or to learn something, or to do whatever it might be that gets us one step closer to that. I'm all in.

[Unknown]

20,000 Hz is produced out of the sound design studios of Defacto Sound. Hear [e-mail] this episode was written and produced by Nicholas Harder and Casey Emerling.

[Sarah Herlinger]

With help from Grace East.

[Dallas Taylor]

It was sound design and mixed by.

[Unknown]

Jesus Certeaga and Brandon Pratt.

[Dallas Taylor]

Thanks to our guests Sarah Herlinger, Deidre.

[Unknown]

Kaldbeck, Ron Huang and Eric Tresky. And thanks to everyone at Apple who invited me in and made this episode possible. Now, accessibility is something that Apple is very open and responsive about.

[Sarah Herlinger]

We are very open to feedback from customers and had a lot of ways for people to be able to communicate directly with our team.

[Dallas Taylor]

If you want to help drive these.

[Unknown]

Efforts forward, you can email the Accessibility team [e-mail] I'm Dallas Taylor. Thanks for listening.

[Dallas Taylor]

Hey, before we go, a quick reminder about our Sound off story competition. We're looking for memorable and surprising sound related stories that are between two and five minutes long. They should be fully produced to the best of your ability, including music, sound, design and credits. The winning stories will run right here on 20,000 hertz sometime this summer. The top creator will also get a private feedback session with our team, plus $100 in 20,000 Hertz store credit. To get the full details, visit 20K.orgSoundOff submissions close on May 7th. That's 20K.orgSoundOff. You can also find the link in the show notes.