← The Economics of Everyday Things
The Economics of Everyday Things
90. Closed Captions
00:19:05
It takes a highly skilled stenographer — and some specialized equipment — to transcribe TV dialogue in real time at 300 words per minute. Will A.I. rewrite the script? Zachary Crockett tries to keep up.
Loading summary
Transcript67 lines
- [00:00]
Ryan Reynolds
Are you still quoting 30 year old movies? Have you said cool beans in the past 90 days? Do you think Discover isn't widely accepted? If this sounds like you, you're stuck in the past. Discover is accepted at 99% of places that take credit cards nationwide and every time you make a purchase with your card, you automatically earn cash back. Welcome to the now it pays to Discover. Learn more@discover.com credit card based on the February 2024 Nelson Report Ryan Reynolds here from Mint Mobile.
- [00:30]
I don't know if you knew this, but anyone can get the same Premium Wireless for $15 a month plan that I've been enjoying. It's not just for celebrities. So do like I did and have one of your assistant's assistants switch you to Mint Mobile today. I'm told it's super easy to do@mintmobile.com.
- [00:46]
Switch upfront payment of $45 for 3 month plan equivalent to $15 per month Required intro rate first 3 months only, then full price plan options available, taxes and fees extra. See full terms@mintmobile.com.
- [01:02]
Zachary Crockett
Katie Ryan's home office in Pittsburgh, Pennsylvania is pretty run of the mill.
- [01:07]
Katie Ryan
I just have a regular Ikea desk. I have a big TV up on the wall. I have a laptop stand with my laptop on it and then I have a monitor stand that has two monitors on it. There's a blanket on the floor for my dog, you know.
- [01:24]
Zachary Crockett
But the work she does at this desk is seen by millions of people every week.
- [01:31]
Katie Ryan
I've done the Super Bowl a handful of times. I've done the Olympics many times. I just did the Oscars a couple weeks ago. Any sporting event that you can think of, I've probably done it. Any major news event that has happened, I have probably been involved in that somehow. Presidential funerals, presidential debates. I remember when the Boston Marathon bombing happened. The breaking news was just constant. I think I was on the air writing without a commercial break for something like three and a half hours.
- [02:03]
Zachary Crockett
Ryan is a captioner. She writes the text transcripts that appear on your TV screen when you turn on closed captioning. She does this in real time.
- [02:13]
Katie Ryan
Most people think their TV just does it. They don't realize that there's a person like me sitting in a room with headphones on and people don't realize that it's happening live. Like if I'm writing a news broadcast or a sporting event, maybe I have like 5 seconds extra than you do when you're hearing it and I have to write it at the same time and try and Keep up with all the speedy talkers that are out there.
- [02:38]
Zachary Crockett
In some ways, it's a good business to be in. One survey found that 50% of Americans and 70% of Gen Z viewers say they watch content with captions on most of the time. But the industry is also rapidly changing. The nimble fingers of human captioners like katie Ryan are up against the neural networks of artificial intelligence services.
- [03:07]
Doug Karlovitz
Technology is the key to the future of captioning, but you know, you need people that are looking at the content.
- [03:18]
Zachary Crockett
For the freakonomics radio network, this is the economics of everyday things. I'm zachary crockett. Today, closed captions. The term captions is often used interchangeably with subtitles, but the two are different. Subtitles are used for translation. Captions are designed for people with hearing impairments, and they describe every auditory element. Dialogue, sound effects, music, and sometimes even background noises.
- [03:50]
Doug Karlovitz
The goal of captioning is to give the user the content of exactly what's being heard.
- [03:58]
Zachary Crockett
That's doug karlovitz. He's a general manager at verbit, the largest provider of captions in america. He says that if you're watching something on tv, either live or pre recorded, you can almost always turn on the captions in the device's settings. But that wasn't always an option, really.
- [04:17]
Doug Karlovitz
Captions were born for television in 1970. The first pre recorded show ever captioned was the french chef with julia child.
- [04:31]
Zachary Crockett
The earliest efforts were called open captions and they were limited to pre recorded shows. The text was a permanent part of the video. Eventually, a new method called closed captions made it possible for viewers to turn the text on and off. And by the 1980s, thanks to the efforts of the nonprofit national captioning institute, captions could also be used for live television. Around this time, Karlovitz's father, Joe, saw an opportunity to expand the captioning industry.
- [05:04]
Doug Karlovitz
My father was a court reporter, a stenographer, and he became very interested in computers and how to take his stenotype and get it translated through a computer into english.
- [05:19]
Zachary Crockett
Stenographers are extremely fast typists on stenotype machines. They can Transcribe up to 300 words per minute. Joe began training fellow stenographers to do TV captioning, and in 1986, he founded a company called vitac, which was later acquired by another company called verbit.
- [05:40]
Doug Karlovitz
We started out with a local television station in pittsburgh and eventually grew into the largest provider in north america of captioning.
- [05:51]
Zachary Crockett
Today, broadcasters, cable companies, and satellite services are required by federal laws to have Captions available for nearly every televised program. This also carries over to much of the media on streaming services, online and most video content in public settings like courtrooms, hospitals, schools, and sports bars. Captions have to be readable, accurate, and inclusive of all audio context. They have to clearly identify each speaker. And for live broadcasts, like news programs, they appear almost in real time.
- [06:26]
Doug Karlovitz
In the United States, everything that airs on television should have captions. Today, almost every show has captions on it.
- [06:37]
Zachary Crockett
Fitech is one of three companies alongside IBM and Zoo Digital Group that control around 60% of the captioning market. Karlovitz says they caption around 500,000 hours of content a year.
- [06:51]
Doug Karlovitz
We work with all the major broadcasters, all the various producers of television programs, work with all the different universities around the world providing captions for the classroom. On the legal side, we're working with local law firms and court reporting agencies. And on the government side, we'll do anything from town halls to training on all the different things. We also work with sports venues, theaters. So everywhere where words are spoken, there's the opportunity to add captions.
- [07:28]
Zachary Crockett
Much of today's captioning has shifted from human stenographers to automated tools. In some cases, the captioning service uses a technique called re speaking. A human employee watches a show in a recording booth and carefully recites every word into a special microphone. Voice to text software turns the narration into a written transcript. In other cases, particularly with pre recorded TV shows, technology can be used to generate text from a script. But for live tv, like news broadcasts, super bowls, and presidential debates, a human captioner clacking away at a machine is still the most reliable option. A stenographer gets a live feed of a network's audio a few seconds before it goes to the general public. They listen through a pair of headphones while typing out the words in shorthand on their stenotype machine. This shorthand goes through processing software on a computer that turns it into text. The text is embedded in a video signal that's transmitted to the television network through modems and IP connections. And when you press the closed captions button on your remote, a microchip inside your TV retrieves and displays the captions on screen. It's a complex process, and networks might pay verbit anywhere from 130 to $175 per hour for live human captioning services.
- [08:56]
Doug Karlovitz
So if you have a broadcast show that's in a 30 minute block, but it may be really only on the air for 24 minutes, they would pay for that on a per minute basis. If you're doing a live show, you're paying basically for the times that are booked because you don't know how long those live shows can go.
- [09:16]
Zachary Crockett
So who are these humans who create the captions on tv? And what's it like to be on the clock during a live broadcast?
- [09:25]
Katie Ryan
Sometimes you can't even get a drink of water.
- [09:27]
Zachary Crockett
That's coming up.
- [09:31]
Ryan Reynolds
Are you still quoting 30 year old movies? Have you said cool beans in the past 90 days? Do you think Discover isn't widely accepted? If this sounds like you, you're stuck in the past. Discover is accepted at 99% of places that take credit cards nationwide and every time you make a purchase with your card, you automatically earn cash back. Welcome to to the now it pays to Discover. Learn more@discover.com credit card based on the February 2024 Nelson Report.
- [10:00]
Zachary Crockett
The Economics of Everyday Things is sponsored by Shopify. When you think about businesses growing their sales beyond forecasts, you think about a product with demand, a focused brand and influence driven marketing. But an often overlooked secret is actually a business behind the business making selling and buying simple for millions of businesses. That business is Shopify. The not so Secret Secret Shop Pay can boost conversions up to 50%, meaning less carts going abandoned and way more sales. So if you're into growing your business, your commerce platform better be ready to sell wherever your customers are scrolling or strolling on the web, in your store, in their feed, in their and everywhere in between. It's no secret businesses that want to grow grow with Shopify. Upgrade your business and get the same checkout that many other companies use. Sign up for your $1 per month trial period at shopify.comeverydaythings all lowercase go to shopify.com everydaythings to upgrade your selling today. Shopify.com everydaythings.
- [11:12]
Ryan Reynolds
Ryan Reynolds here from Mint Mobile. I don't know if you knew this, but anyone can get the same premium wireless for 15amonth plan that I've been enjoying. It's not just for celebrities. So do like I did and have one of your assistant's assistants switch you to Mint Mobile today. I'm told it's super easy to do@mintmobile.com.
- [11:30]
Switch upfront payment of 45 for 3 month plan equivalent to $15 per month Required intro rate first 3 months only, then full price plan options available, taxes and fees, extra fee@mintmobile.com.
- [11:46]
Zachary Crockett
Katie Ryan didn't start out hoping to be a professional captioner.
- [11:51]
Katie Ryan
When I was graduating high school, I really didn't know What I wanted to do with my life and my great aunt Sandy. Her sister at the time was an official court reporter in Philadelphia. And Sandy said, well, you can type fast on a keyboard. Why don't you look into stenography?
- [12:10]
Zachary Crockett
Ryan completed a court reporting program at a community college in Pittsburgh and joined vitac, now Verbit. After graduating, she's been at the company as a captioner for more than two decades. In her work, Ryan uses a machine called a stenotype. It has a small screen and around 20 unmarked keys that look kind of like popsicle sticks. She's able to type at speeds of up to 300 strokes per minute. Using a technique called chording, she presses down on multiple keys simultaneously to phonetically spell out whole syllables, words, and phrases with one motion.
- [12:48]
Katie Ryan
Stenography is essentially learning another language. It's combinations of keys to make words, and so on the machine, each key has a letter, and then there are combinations of keys that make more letters. PB would be N. The letter I would be eu. The letter D would be TK and then there are combinations of keys that make words. So and would be a, P, B, D. Your hands are on different sides of the keyboard on the machine. Your left hand is prefixes, your right hand is suffixes. And then you have your endings. I, N, G, S, E, D. On your right side, Ryan can spell out.
- [13:34]
Zachary Crockett
Entire phrases with just a few keystrokes.
- [13:38]
Katie Ryan
A good example would be like, ladies and gentlemen, that would be good for TV or court. On my machine, it would be L, A, I, R, J. So you hit all of those keys at once, and, ladies and gentlemen, will come out in your computer software in one fell swoop. In one stroke, you get all of those words.
- [13:56]
Zachary Crockett
Before she goes live, Ryan creates a dictionary full of customized briefs, abbreviations of specific words that she knows will reoccur throughout the broadcast. For the Academy Awards, she'll program combinations of keystrokes for the title of each nominated movie. For a hockey game, she'll program every player's name instead of having to write.
- [14:18]
Katie Ryan
Out their name every single time that it's said. You hit that one combination of keys one time or twice, and then that whole name will come out. Obviously, we have to search ahead of time to find out who, like, your play by play announcer is and who your color analyst is.
- [14:34]
Zachary Crockett
But the process doesn't usually go without a hitch or two. Captioners are human, after all, and they make the occasional mistake. While there's no federally mandated benchmark, the standard for accuracy in the industry is 99%, meaning one out of every 100 words might be misspelled or altogether butchered. Oftentimes a captioner is aware of a typo. They just don't have the time to fix it. During a high speed live broadcast, we.
- [15:04]
Katie Ryan
Have the asterisk on my machine, which is the key in the very middle that can erase a mistake. But nine times out of 10, you are not going to catch it fast enough before it already goes out on the air. And then if you try and take it back, it's just going to garble the captions up. So it's better to just, if you make a mistake, just ignore it and keep writing and move past it. And then the faster it moves off the screen, the faster people will forget about it.
- [15:28]
Zachary Crockett
Even after 21 years on the job, Ryan has a few recurring issues.
- [15:33]
Katie Ryan
I tend to drag my fingers, so sometimes I will catch extra letters when I'm trying to write. Certain words miss keys too. Like if my fingernails are too long, sometimes I can't quite hit the keys right.
- [15:48]
Zachary Crockett
Sometimes you might notice the captions pause for a few moments or go blank. This is likely because the captioner fell off pace and is trying to catch up. This happens most often with news shows where the banter can be lightning fast. Rachel Maddow, who hosts her own live show on MSNBC, has been clocked talking at up to 270 words per minute, a challenge for even the most seasoned captioner.
- [16:15]
Katie Ryan
If you need to just let a sentence go and then catch up again, that's okay. When you start paraphrasing though, then you take the risk of presenting the wrong information or turning it into something that they didn't actually say. And that's the last thing you want to do. You don't want to put words in anybody's mouth.
- [16:31]
Zachary Crockett
The goal is to provide a text equivalent of as much of the audio as possible. This can be particularly challenging when multiple people are speaking at once.
- [16:42]
Katie Ryan
A lot of times it'll just be, you know, a couple of words and a dash, and then the next person it'll be a couple of words and a dash. Sometimes there's nothing you can do. If they're just screaming at each other, there is nothing you can do. You know, once they figure it out, then you can keep going again.
- [16:56]
Zachary Crockett
Doug Karlovitz, the general manager at Verbit, says certain TV shows pose more problems than others. Like the Osbournes, a reality show from the early 2000s that followed the aging and often incomprehensible rock star Ozzy Osbourne and his family.
- [17:17]
Doug Karlovitz
The debates around the office on what we thought he was saying on that show was, you know, good water cooler conversations. Well, first was, is he just putting this on? Eventually, as that show got renewed, you realize, no, that's how Ozzy talks. It was really like, I think he said this and then, you know, people would go and come over, listen to this. What do you think he said? And you know, you would just sit there and I don't know, I don't know what he said. I don't think he knows what he was saying.
- [17:48]
Zachary Crockett
There are also elements that require interpretation, like how to caption a noise or a non verbal vocalization. Some networks and studios are particular Disney reportedly has specific rules about how R2D2's mechanical noises should be captioned. Netflix is fond of using the phrase wet squelching to describe the sound of monsters in the show Stranger Things. For background noises and live captioning, Ryan uses a list of templatized descriptions.
- [18:19]
Katie Ryan
We call them parentheticals. So like bells tolling or applause, singing, chanting, things like that, you want to try and be descriptive, but also you don't want to go overboard.
- [18:31]
Zachary Crockett
All of this effort is to ensure that people who are deaf or hard of hearing have equal access to media. But captions have found a much broader audience. A 2022 survey by the language learning platform Preply found that half of all viewers now watch media with captions on most of the time. Some have speculated that's at least partly to do with modern sound mixing, which alternates between loud sound effects and quiet dialogue.
- [18:59]
Doug Karlovitz
Game of Thrones there was so much background noise occurring on that show that a lot of the people started using captions.
- [19:07]
Zachary Crockett
But the most frequent users of captions are now younger people, particularly Gen Z. And that has more to do with changes in the media landscape.
- [19:18]
Doug Karlovitz
The younger viewers, they're watching it on their phones, they're watching it on their iPads. They're not necessarily listening, but they're reading it as they're in class or they're at work and don't want to call attention to themselves.
- [19:31]
Zachary Crockett
Some publishers have estimated that up to 85% of the videos they post on Facebook are watched on mute. Many short form videos on social media sites now have captions coded directly into the media file that can't be turned on or off.
- [19:47]
Doug Karlovitz
That's because it's keeping that person who's looking, it's keeping their attention longer.
- [19:54]
Zachary Crockett
Some platforms like YouTube offer their own tools to creators that use speech recognition to generate captions automatically. Karlovitz says artificial intelligence has already fundamentally changed the captioning business. Verbit offers automatic speech recognition and generative AI tools that are trained with diverse language models to pick up on speech patterns. Karlovitz says these options cost much less than traditional transcription, but they still aren't as accurate or precise as a human captioner. And at least for now, many clients still prefer their captions to be generated by a human being like Katie Ryan.
- [20:38]
Katie Ryan
Maybe a deaf person is in an area that there's tornadoes and they turn on their local news. We want those people to be able to have captioning that is as accurate and as clean as possible so they know what to do and they can be safe. I will always advocate for a human captioner to be there to give the best service possible.
- [20:58]
Zachary Crockett
When you watch tv, do you always use the captions?
- [21:01]
Katie Ryan
No. Never have captions on in my house.
- [21:04]
Zachary Crockett
Really?
- [21:04]
Katie Ryan
Never? No. I sit in front of a computer and deal with that all day. I don't need to worry about it. I'm off the clock.
- [21:18]
Zachary Crockett
For the Economics of Everyday Things, I'm Zachary Crack. This episode was produced by me and Sarah Lilly and mixed by Jeremy Johnston. We had help from Daniel Moritz Rapson and thanks to our listeners Owen Roberts and David Kennett for suggesting this topic. If you have an idea for an episode, feel free to email us@everyday thingsreconomics.com Our inbox is always open. All right, until next week. What if you're in the middle of like a live broadcast and you just really have to pee?
- [21:56]
Katie Ryan
Now from my office to my bathroom is like 10 steps so I can make it the Freakonomics Radio Network the Hidden side of Everything.
- [22:12]
Zachary Crockett
Stitcher.