Loading summary
Transcript1 lines
- [00:03]
A
Here's an idea. A machine that translates sounds into electrical signals and then stimulates the auditory nerves to send that sensory data to the brain. Can it work? And can it convey actual words? For many years, people didn't believe it. Eminent auditory scientists like Dr. Merle Lawrence stated that speech comprehension was not feasible. The inner ear was too complex, too many nerves. We knew too little about how it all works. And the surgical trauma and electrical stimulation will kill the nerves anyway. And yet, they did it. And now hundreds of thousands have had their lives improved with this device. In this video, the long and winding road to the cochlear ear implant. Hearing happens because of special hair cells inside the ear. Yes, no joke. In a healthy person, sound waves entering the ear travel through the outer ear and vibrate the eardrum. The eardrum is connected to three tiny bones inside the middle ear called ossicles. The ossicles turn those sound waves into mechanical vibrations, which are then transmitted into the inner ear, or the cochlea. The cochlea is shaped like a snail, about 10 millimeters wide and is full of fluid. The mechanical vibration travels through this fluid, creating a pressure wave that disturbs this membrane called the bacillar membrane. Attached to the membrane are bundles of hair, like things called stereocilia, which come out of hair cells. There are two types of cells. Outer hair cells amplify the membrane's displacements for the inner hair cells, which are the main listeners. When the movements hit these inner stereocilia, they bend and open tiny ion channels. That leads to an auditory nerve signal that the brain interprets as sound, music, or language. You know, I actually had no idea that that was how it worked. And it's quite elegant, too. The basilar membrane codes to a gradient of sound frequencies. Higher frequencies near its base, where it's narrow and stiff. Lower frequencies near the top, where it's wide and floppy. Anyway, humans are born with about 15,000 outer and inner hair cells, but that number declines throughout life because of drugs, infections, trauma, or loud noises. Unfortunately, the cells do not grow back much as like the hair cells on some people's heads. When enough inner hair cells die, you get some hearing loss. For those people, hearing aids are to make it easier for the remaining hair cells to pick up sound. But that requires at least some of those hair cells to be alive. If the cochlea and its cells are totally damaged, then we are out of luck. Yet if the central auditory system is still intact, would directly stimulating the nerve sidestep those missing hair cells and let certain deaf people hear again. And how might we do that? The first connections between auditory signals and electricity emerged in the second half of the 1700s. Then, in 1800, Count Alessandro Volta, the Italian chemist and electricity physicist known for inventing the battery, placed two ends of his 50 volt. Oh, is that where the unit comes from? Battery to his ears. What happened next will shock you. Volta heard a crackling and boiling and then an artificial hiss and boom. In other words, he quote unquote heard. Heard direct electrical stimulation of the auditory nerves. He later wrote, the disagreeable sensation, which I apprehended being dangerous, of shock in the brain, prevented me from repeating the experiment. A few brave ones have since repeated this experiment. Throughout the decades, some even tried to treat deaf or hard of hearing people with electricity, with difficult results. In the 1930s, progress was made in sussing out how the ear converted acoustic waves into electric signals. In 1929, Ernst Glenn Weaver and Charles Bray at Princeton sedated a living cat and connected its auditory nerve to a receiver, a telephone receiver, via a telephone wire. One person then spoke into the cat's ears, while the other person, sitting inside a soundproofed W box, read the sound coming through the telephone receiver via the ear. They found that if they spoke louder into the cat's ear, the output signal got stronger, too. The nerve was copying the sound, preserving the wave's physical shape. It was early proof that the ear was encoding sound waves into bioelectric actions. Weaver would spend his life studying hearing in various animals, from little lizards to big dolphins, writing tomes titled the Reptile ear and Amphibian Ear. What a legacy. In 1937, Harvard, Stanley Smith Stephens and his team placed special electrodes inside the middle ears of patients who had their eardrums and ossicles removed. He discovered that an AC current through or around the head created an auditory sensation, naming this phenomena electrophonic hearing. This got people looking into whether they can help the deaf hear by directly stimulating the auditory nerve, using different electrodes and signals to produce tones or sounds. In the 1950s, doctors practicing a branch of medicine called otology began investigating deafness. Otologists treat ear diseases primarily through surgery with antibiotics. Now suppressing traditional illnesses they addressed, they turned to deafness. In 1957, two French ontologists, Andre Giorno and Charles Irieer, made auditory history. The Algerian born Giorno was an electrophysiologist who had built a career out of researching how electricity affected the human body. In the mid-1950s, he built a prosthesis that stimulated motor nerves as a possible electrical alternative to the iron lung for polio sufferers. In 1954, Giorno starts talking about using something like this for hearing. From whom or where this idea came from is not too clear. Giorno has said different things over the years, but he works on it for the next few years. In 1957, he met another otologist named Charles Arrier, who was then about to operate on a particularly unfortunate 57 year old man. This patient suffered a locally destructive growth inside his middle ear. Previous surgeries for this growth had damaged the patient's cochlea and labyrinth, the latter an inner ear system for balance. His facial nerves had also been cut, leaving his face severely paralyzed. To help restore some facial function, Irie was going to do a facial nerve graft operation. Giorno suggested also implanting his new prosthesis to try and stimulate the auditory nerve to grant some hearing. Iryea agreed. The graft was successful, but when Airye tried to insert the prosthesis, he found it a bit challenging. The cochlea had been destroyed, leaving behind only a cavity. Just thinking about that makes me cringe a little. So they looked around for another suitable place to attach the prosthesis, and eventually they found a small stump at the end of the frayed eighth cranial nerve. This stump wasn't inside the cochlea, but rather an adjacent region where the auditory nerves bundled together with other nerves handling various balance functions before heading off to the brainstem. The prosthesis was a crude device that transformed sound waves into electrical pulses. Giorno made it himself, by hand. It worked in that the patient heard something he characterized as sounding like cicadas or whistles. He never was able to comprehend normal speech, but did learn to recognize certain sounds, like mama or papa. But after a few months, the device broke. They did a second operation which had a similar ending, and this led Irie to question the device's build quality. Giorno, you see, was an idealist. He strongly believed that science should be made a public good for all. So he never patented any of the devices or procedures associated with his work. Nothing wrong with that. But this made it hard to raise funding for a better quality prosthesis. Giorno refused Irie's request to approach a company leading the latter to turn down a third surgery. Further experiments did not work out, for reasons I won't get into, but had little to do with the science. And by 1959, Giorno was ready to move on to something new. He felt his role was to open up the space for others, and that job was done. As Giorno was stitching things up, someone brought A French newspaper clipping of his work to an ontologist at the Ear Research Institute in Los Angeles named William F. House. House was astonished at the results he was reading. He before assumed that any auditory stimulation required an attacked cochlea. Furthermore, he recalled, the amazing thing to me is that Giorno and Iriea never published any more on it, nor did they do more than, I think, two patients. That stimulated a lot of interest on my part. So Hao started experimenting with patients during certain ear surgeries, putting on electrodes and asking patients if they heard anything or felt uncomfortable. The results were encouraging enough to find some volunteers. So Dr. House collaborated with a neurosurgeon named John Doyle and his brother Jim, an electrical engineer, on a device. It established many of the key a microphone, a signal processor to turn sound into electrical signals, and electrodes to be implanted into the cochlea. This device was first implanted in January 1961. It had a single gold electrode that made contact with the cochlea through an incision in the skin. This patient experienced auditory feelings, but later fears of infection caused Haus and others to take it out. A month later, they tried again with something new. As I mentioned, different spots at the cochlea correspond to different pitch ranges, like a keyboard. So for a richer sound, all those spots need to be stimulated. So Haus put in a system with five electrodes touching various areas spaced out about 20 millimeters along the membrane. It was powered by an induction coil transmitting power through the skin. With this, the patient could hear different tones, but the skin got still too irritated, so the device had to be taken out again before further work can be done. However, their collaboration ended. House later said this was because the Doyles prematurely shared news of these experiments with the press, encouraging what House felt was unreal hype. The Doyles also refused to share their product reports, claiming the work for themselves. So they went their separate ways. The Doyles implanted a few more patients until they ran out of funding, and the disheartened House, meanwhile, paused things for some time. The publicity around Giorno and House's results encouraged others, like Stanford's Blair Simmons. In 1964, Simmons and his colleague, engineer Robert White, implanted an array of six electrodes into an elderly patient. Each of the electrodes were connected to their own wire, giving it far better frequency differentiation. While presenting his work, Simmons started formally calling this a cochlear implant. He had already been floating around informally, but this made it official. Blair's patient reported different auditory sensations when certain waves were stimulated, but then unexpectedly lost his vision. Without that, he could no longer read lips and help the work. The device was then removed for infection reasons. Then, in late 1960s, devices like the first implantable pacemaker had emerged with plastics, epoxy and titanium encapsulation to keep electronics and wires away from human fluids. Transistors also helped make them smaller. These smaller, more biocompatible systems inspired Dr. House to try again. In 1969, he and his new partner Jack Urban came up with a longer lasting six electrode system. These systems, as well as one with a single electrode were implanted in a few patients. In 1979, House presented the results of the first long term study of these implanted patients at the first International Conference on Electrical Stimulation of the Acoustic Nerve as a Treatment for Profound Sensor Neurodeafness in Man at the University UC San Francisco, UCSF and Stanford had their own group working on clinical implants at the time, led by Dr. Robin Mickelson. They and the House LA group were seen as the leaders in the field anyway. House's presentation noted that patients could not discern between the pitches when multiple electrodes are stimulated, perhaps due to cross interference. He recommended going to single electrode for now, but nevertheless declared the electronic cochlea ready for more widespread testing and development. Prior to the 1973 UCSF Conference, few in the auditory community had heard about these implants, or if they had, it was dismissed as perilous quackery or a fruitless venture. The conference validated the core concept and started talk on a set of shared safety procedures presented with results including animal tests. The National Institutes of Health in 1975 commissioned a landmark study from Robert Bilger of the University of Pittsburgh to evaluate 13 human recipients of cochlear implants. Their report, published two years later in 1977, concluded that while the devices were safe and can aid in lip reading or identifying sounds in the environment, they cannot help with open set speech recognition. This cemented the product's legitimacy while also throwing down the gauntlet for the next big goal. Can these things get good enough to help people understand speech? As the news traveled the world, new programs emerged to pursue this technology. In France, you had a team led by Claude Henri Chouar, an ontologist at the St Antoine Hospital in Paris. Chouar was a student of the French surgical pioneer Irie and had long dreamed of a James Bond style electronic device that can alleviate the considerable handicap of deafness. After attending the 1973 San Francisco conference, he worked on a multi electrode device that can sample speech across multiple frequency bands and independently stimulate the cochlea at various points along its length. It was later patented by him. He recruited a small French electronics company called Bertin to produce a multi electrode device. While it was a serious contender, Schouard did six implantations of a seven electrode device across 1976 to 1977. The implant itself was large and cumbersome. Its complicated and time consuming surgical insertion procedure hampered adoption. Later devices improved on this, though. Another major European pioneer starts with the couple Ingeborg and Erwin Hochmaier from the Technical University of Vienna. In the mid-1970s, they were asked by a clinic to design a cochlear implant. By December 1977, they built an eight channel electrode device that sampled sound, split it into eight frequency bands and then fired pulses corresponding to those at a rate of 10kHz. Quite advanced, the concept behind the eight electrode device was sound. But it did not work quite well in practice. The eight channels suffered too much crosstalk. Subsequent speech tests found better results with devices having fewer electrodes and channels. The Hochmayers also pivoted to analog broadband processing, taking the incoming sound's whole wave, its broadband, and passing it all through at once to the nerve as a continuous analog signal. It worked well enough for patients to sync what they heard with visual lip reading at a time when the processing algorithms and microelectronics were not yet quite ready. These approaches kept the devices simple and practical, but no speech perception. So the white whale surfacing throughout this whole narrative has been the multi electrode implant. Time and time again, various people have pursued it. William House in 1961, Simmons in 1964, House again in 1969, Schwarr in 1973 and the Hochschmers in Vienna. These efforts did not grant speech recognition, and by then the community was split between good enough single electrode devices and multi electrodes. In the end, the first commercial multi electrode device would be brought to the market by the Australian Graham Clark and his team. Clark was inspired as a child to become an ear surgeon by his deaf father. In 1967, he read about the cochlear implant work done by Simmons and resolved to take a crack at the speech recognition problem. He started by trying to figure out whether the brain can even recognize electrical signals as sound. Using a series of animal tests in the late 1960s at the University of Sydney, Clarke deduced a few insights about electrical stimulation and hearing. Importantly, Clarke found that neurons have a speed limit in terms of how fast they can handle stimuli. About 200 to 300 pulses per second, maybe 500 for a bit. Anything faster is either suppressed or just turns into a muddle. This realization told Clarke that any implant for speech had to have multiple electrodes. Understanding human speech required something like 3,000 pulses a second. But if an individual nerve can only handle up to 300 pulses a second, Clarke would need to put electrodes at different spots along the cochlea, while also ensuring that their electrical activity did not interfere with one another. And since the device cannot pass through the entirety of the speech to the nerves, Clarke also realized that there was a bottleneck. So his machine needed to determine the essentials of speech and pass on only those. In 1973, Clark partnered with a physicist friend named David Dewhurst to prototype his bionic ear. This device used chips interconnected and packaged inside a box and that can resist the conditions of the human body. It had to be small and made from a chemically inert material. And like others before, it received power and data through radio signals sent through the skin. The wafers were fabbed in America, but they needed to be wired together and put onto something like a circuit board, but implantable rather than a usual circuit board. Clark and his team wanted to use silica wafers, which with printed on gold wires. To produce these silica wafers, he consulted awa, the iconic Australian electronics company I covered earlier, as they had produced pacemaker components before. But they declined, saying that the risk was too high. So Clark turned to Australia's version of Bell Labs, the research labs of Telecom, now Telstra. And to their credit, the lab did it for free, sending over the completed wafers a few weeks later. The silicon chips then had to be bonded, welded onto the silica wafers and sealed inside their gold plated box. This was done by a small company called Hybrid Electronics. The closed box is then wired to an electrode and sealed off with silicone rubber. This device array had 20 platinum electrodes, more than prior devices made by Simmons and House. This brought up questions about whether so many electrodes can be inserted into the cochlea without serious damage. A long running concern. The fluid filled channel inside the cochlea, called the scala tympani, is just one to two millimeters wide and very delicate. Various animal tests and models failed to get the electrodes deep enough inside this spiraling chamber. Then in 1976, whilst at the beach, Clark had an epiphany as he collected and played with these turbine shells shaped like the cochlea. He discovered that blades of grass and fine twigs can easily enter and follow along the curve of the shell. So he designed the electrode Array the same way, flexible at the tip, but increasingly stiffer, moving towards the base. This was done by incrementally adding wires to the banded electrode array. This all required money and Clark struggled to get funding, all in these early days. The Australian government's premier medical research granting body granted a few thousand dollars in 1971 and 1972, but issued no more until years later when the concept was proven. Frankly, they didn't think it would work. Multi electrode systems seemed too complex considering the engineering knowledge of the era. Single electrode systems showed more promise in the mid-1970s and of course scientists remained skeptical of the entire field. Until the 1977 NIH report came out. Clark was proposing to put electrodes inside the cochlea, a vulnerable area. His own peers warned that the device can further damage nerves or even kill someone. They called him that clown. Clark Graham nevertheless kept going, raising money with paid speeches, community donations, TV telethons, and at one point literal street donations. Finding a suitable subject had been difficult. Few of his peers wanted to refer their patients to him. Clark recalled in his fine book Sounds from Silence, which provides many of the specifics in this section. First vetting one earlier patient who lost her hearing from meningitis, but eventually turning her down due to bone growth inside the inner ear. In April 1978, Clark was approached by 46 year old Rodney Saunders, a supervisor for a grain merchant. Saunders lost his hearing two years prior in an accident. After it happened, he could no longer do his job and stayed home most of the time. He was a poor lip reader and wanted to hear again. The surgery was done in August 1978 and went without a hitch. After he healed, he started coming to the lab to connect his implant to a large audio processing computer for testing. The early results were discouraging. No hearing restoration. All Saunders said he heard were the hissing noises characteristic for the deaf. Before the third appointment, however, Clark's team discovered a flaw in the testing equipment. Upon fixing it, Saunders said that he heard sounds. They later played songs like God Save the Queen and Waltzing Matilda, which I'm told is a somewhat famous Australian song, and Saunders recognized those immediately. Clark announced the successful implantation to the Press in mid September 1978 and the news traveled around the world. A second patient, George Watson, got the implant and the first lady, who had meningitis, was operated on. But overgrown bone in her ear unfortunately caused it to fail. While Watson and Saunders implants demonstrated the concept, the systems themselves were big, impractical and unreliable. Hearing sound required connecting to a large computer in a laboratory And a few months after Watson received his implant, it broke. More work and money were needed to make a real product. So the team applied to the Australian government for a grant. The government applied a lot of scrutiny, even evaluating Clark's marital situation and requiring written rebuttals to a random Australian professor who claimed that Stanford was already doing this. So there was little point for Australia to do the same. But finally, in January 1979, the Australian government issued a 2 year 1 million AUD grant to commercialize the implant. With the money, Clark was able to hire new workers. A notable one was the experienced electrical engineer Peter Seligman, who worked with a colleague named Jim Patrick to shrink the speech processor from a trolley sized rack and to something the size of a binoculars case. The PSP one was small enough for two patients to walk out with them, a remarkable achievement. After achieving the phase one milestones, Clark's team needed a company partner to develop the clinical system Clark had approached. A few companies like The American firm 3M 3M wanted to enter healthcare and had earlier licensed Dr. William Houses and older single electrode implant. In the end, they passed on Clark's work to focus on that. Fortunately, Clark had by then built a relationship with Paul Traynor, owner of an Australian company called Nucleus Limited and today seen as the father of the Australian medical device industry. Nucleus owned a company called Telectronics, today known as a pioneer in cardiac pacemakers. This expertise, as well as their Australian heritage, made them the ideal partner for Clark. In 1981, the Australian government issued another $2 million of funding, allowing Nucleus to hire its first employees for the internal Cochlear project. Seligman, Jim Patrick and others joined this team led by a fellow named David Money. Later on, the team became the company cochlear. There they completely redesigned the system, including the sound processor. Seligman and Patrick worked with Peter Singel and a Fabless semiconductor maker called Austech to design a custom ASIC to do the sound processing. The chips worked brilliantly. This final device, later called the Nucleus 22, featured 22 evenly spaced electrodes on a tapered band. It connects to a small receiver unit placed under the skin beneath the ear. An external microphone picks up sound at the ear level, translates it and sends it through the skin to the receiver beneath it. In 1982, the Nucleus 22 was ready for clinical trials in Australia, Europe and the United States. They began recruiting patients and ear surgeons for implantation, hoping to gather enough data to submit to the FDA. They were in something like a race. In 1984, 3M's product, based on William House's single electrode implant, received FDA approval for adults. It was such a landmark moment that the deputy commissioner marked it by saying for the first time, a device can to a degree replace an organ of the human senses. But the 3M house device only had a single electrode and and with that can only bring an awareness of sound. It did not convey speech. In October 1985, the FDA reviewed the data from 87 patients and approved the Nucleus 22, noting that some patients can use it to understand speech without lip reading aid, something the single electrode devices could not do. Clark's pioneering work helped propel the rest of the industry. Back in 1981, 3M licensed the Hochmayer's cochlear implant device alongside Dr. House's and employed them as consultants. But in the late 1980s, 3M exited the Cochlear implant market, unable to Compete with the Nucleus 22 in 1989, the Hohmeyers founded their own company called MEDL, believing that new processing algorithms can help improve the multiple electrode product. One issue with multiple electrode systems of the day was crosstalk Firing electrodes simultaneously inside the fluid filled cochlea caused the currents to interfere with one another. But in the late 1980s, Blake Wilson of the Research Triangle Institute proposed a new, potentially revolutionary algorithm called continuous interleaving sampling, or cis. It does not try to copy a sound wave's continuous movement, but rather breaks it up into electrical frames or snapshots. These are interleaved in such a way that neighboring electrodes do not fire simultaneously and interfere with each other, ergo creating an impression of sound like how a sequence of static movie frames create the illusion of movement. Implementing this algorithm was challenging, requiring the hardware to fire thousands of times per second. In 1994, Med El released their product, the Comb 40, an eight channel system capable of firing a total of non overlapping 12,120 pulses per second. Subsequent products have raised that number yet further. Based in the town of Innsbruck in Austria, the privately held Med El became the second largest manufacturer of cochlear implants after Cochlear in Australia. I want to thank the huge number of people who reached out to me with information and kindness about this particular topic. In particular, I want to thank the two Peter Esses, Andrew B. Thiago R and Rob E for their contributions. Graham Clark continued to work on his device and research. In 1990 he redesigned and began implanting the device into children, a situation that received criticism from the deaf community. The issue of whether children should get these implants remains a hotly contested one. In 2013, Blake Wilson, Ingeborg Hochmeir Desoe, and Graham Clark shared the Lasker Award, one of the most prestigious awards in medicine, for their work in pioneering the modern cochlear implant. The cochlear implant industry continues to develop and improve. Nowadays, they can employ a robot to do the insertion. The audio quality has improved thanks to various coding strategies. And. And on the horizon, there is the potential of implants placed totally inside the body, giving patients unprecedented freedom. The work marches on. All right, everyone, that's it for tonight. Thanks for watching. Subscribe to the channel. Sign up for the Patreon, and I'll see you guys next time.