← Design Better
Design Better
Bonus Episode: Inside Sonos's distributed research & design process for the Arc Ultra
00:33:52
We chat with Matt Benatan, Principal Research Scientist, and Naphur van Apeldoorn, Senior Manager, Hardware Development Engineering, about the design and development of their newest product.
Loading summary
Transcript54 lines
- [00:01]
Matt Benetton
On the personal front, quite often we'd be watching content with family members who might be struggling with dialogue and so looking for ways to enable that for people so that they can enjoy content with loved ones is something that we're very happy to have been able to do, and it's one of the key motivating factors behind the R&ID collaboration.
- [00:23]
Aaron Walter
For many creative people, music is a central part of life. You can count me in that camp. I have long admired Sonos for the quality of their products and easy integration of their products into my home, and everything they design seems to be beautiful. The integration of design and technology is often a challenge for companies because they've got different groups with different values trying to work together. But from the outside, it would seem that Sonos has something figured out in this department. To learn more about their approach, we spoke with Matt Beneten, Principal Research Scientist, and Nafur Van Appledorn, Senior Manager of Hardware development and engineering.
- [01:06]
Eli Woolery
We chat with Nafur and Matt about how Sonos decided to invest the time and energy into a new product, Arc Ultra, and what the RD and prototyping process look like. As a distributed team, they share physical prototypes using a 3D printer. We also talked about how they're using onboard AI for speech enhancement and why it seems like movie dialogue has become harder to hear over recent years, even if you don't have kids making noise in the background while you're trying to watch this is Design Better, where we explore creativity at the intersection of design and technology. I'm Eli Woolery.
- [01:38]
Aaron Walter
And I'm Aaron Walter. At DesignBetter, our primary mission is to produce work that helps people like you refine your craft, improve your collaboration skills, and get inspired by the creative process of others. If you enjoy what we do here, the best way to support us is to become a Premium subscriber@designbetterpodcast.com subscribe. We'll return to the conversation after this quick break.
- [02:16]
N/A
Design Better is brought to you by WIX Studio, the platform built for all web creators to design, develop and manage exceptional web projects at scale. Learn more@wix.com studio.
- [02:31]
Aaron Walter
And now back to the show.
- [02:38]
Eli Woolery
Nafoor Van Appledorn and Matt Benetton, welcome to the Design Better podcast.
- [02:43]
Matt Benetton
Thanks for having us.
- [02:44]
Eli Woolery
We're excited to talk to you. Aaron and I are both really big Sonos fans. I've got quite a few of the speakers and Aaron has even more. So many, in fact, that he doesn't know quite exactly how many he has in his house, which is pretty cool let's start with this new product that you guys released. And we want to figure out a few things, I mean, for our audience that's sort of within the design and product design realm. We thought it might be interesting to start from a user research perspective and maybe you could talk to us a little bit about how you did the user research. Also, I believe one of you was prior on the R and D team too, so maybe that was part of it. But how did you decide to invest that time and energy needed to come up with this new product?
- [03:25]
Nafur Van Appledorn
Yeah, so I was part of the team which developed the sound motion transducer in Arc Ultra. And it's really cool how that happened. So I'm not part of Sonos for three years. And that's also when the acquisition happened of like the MITE technology. And basically three days after the acquisition, we were flown to Santa Barbara, one of the headquarters of Sonosys over there, and we joined the room. 30 engineers, user research and all kind of other teams were there and they had like this Arc, the previous sound bar on the table, and they had our transducer technology right next to it. And they were like, okay, how can we fit like this technology inside the sound bar? And we were like, okay, why do we want to do this? And like all those kind of questions, you know. But there was this ambition of creating a new sound experience which was previously impossible without this technology. And then like, what is this technology about? Or the sound motion that's really about creating more bass from a smaller package. That's essentially what it enables you from a technical perspective. But what it does from a product perspective that becomes like very interesting is because you have now kind of dedicated subwoofer or lower frequency device in your soundbar. The other speakers can focus on the vocal range, the placement of audio within the room. And that allowed us to make it like a truly Dolby Atmos soundbar with a 9.1.4 representation inside the room. And that allows you to deliver Dolby Atmos content inside everybody's living room. That's the true innovation here, that we could enable that experience for our home users.
- [04:57]
Eli Woolery
So for folks that might not be familiar with Dolby Atmos, is that what you'd hear if you go to a cinema, like typically go to the movies? That's the type of technology they're using there.
- [05:05]
Nafur Van Appledorn
Exactly. Like this full sound bubble, not only from the front left, right, but also the upper channels. That's this whole Dolby Atmos bubble we can deliver. It's really in the nature of Sonos, of delivering that kind of content to users. If you already go back to where sonos was founded 20 years ago, should realize back then there was no Spotify, Apple Music. You know, music streaming wasn't at the level where it is today. And they decided like, hey, we want to deliver music content to people that are home, everywhere, anywhere, all the time. It started with like the all in one products, but nowadays we have soundbars, wearables, portable devices. We have a whole wide range of products where we can deliver music anywhere to people. And we believe like this Dolby Atmos content is the next content we want to deliver to people. Like you have now Spotify, Apple Music, delivering Dolby Atmos content, you know, and people need devices to play that content as well. It's not only music, but it's also movies, theater. And this is our promise on delivering that experience to people, their homes.
- [06:06]
Aaron Walter
Nufor, you alluded to an acquisition that I'm not familiar with. What was the acquisition that Sonos made that brought this technology into the company?
- [06:14]
Nafur Van Appledorn
So that was the acquisition of Might, which was a startup based in the Netherlands and was founded by two brothers, Matthias and Timothy Schaek. They invented the technology where we can make low frequency speakers five to six times smaller than conventional technology. And that's like the technology which Sonos acquired three years ago. And it's amazing like how that's now available to everybody inside the Arc Ultra product. And to give a reference of what kind of leap in technology we made over there, the previous Arc is and was an amazing product, but that one had like 11 transducers inside the soundbar. And with Arc Ultra we're able to fit like 14 transducers in that single soundbar. And then if you dive a bit deeper, what those transducers consist of. Inside the previous soundbar we had eight woofers and three tweeters. And those eight woofers were responsible for the mid and low frequency and those tweeters for the high frequencies. And because you only had three tweeters and those woofers were also responsible for the bass, it's very hard to position sound as the Dolby Atmos bubble in the room, but also have that vocal clarity which the AI speech extraction is really good with as well. With the Arc Ultra, we have 14 transducers. One is the sound motion transducer based on the mic technology which Sonus acquired. We have six woofers which deliver that mid range frequency and we have seven tweeters which point the sound everywhere in your room. And that gives you that Dolby Atmos content.
- [07:39]
Aaron Walter
Yeah, Matt. So your background is in machine learning and AI. And this is fascinating the way that y' all are bringing that into audio design too, because there's a lot of different reworking and reinterpretation that can happen in the devices to deliver a different type of sound experience. Can you tell us a little bit about how the software inside of devices is playing a role here?
- [08:06]
Matt Benetton
One of the things that came along with arcultra was a bit more processing capability. So before we get to the software, we need to make sure that we have the hardware that we actually need to support the stuff that we want to do. And with arcultura, we had more horsepower. And during the development of arcultra, there was some work going on within what was then the Advanced Technologies team at cenos where we were looking into two techniques for speech extraction. So speech extraction is part of the broader field, if you like, of source separation. And these are the kinds of algorithms that you'll encounter them kind of day to day, particularly if you're on zoom calls, even on telephone calls. These are the sorts of algorithms that remove that stationary noise and provide clear speech signals. And for years and years these have been developed in telecommunications. But applying them within film and television content is much more challenging because in this domain we've got just a much, much broader variety of different types of sounds. And we're also kind of fighting with the fact that these are intentionally crafted. If you're struggling with dialogue and you're watching a film by a well known director, it's going to be designed to be that way. Right.
- [09:21]
Aaron Walter
David lynch, for example. Impossible to hear.
- [09:25]
Matt Benetton
Exactly. And there are many, many examples of this, which I and others have struggled with. And that's before we even get to the point of hearing loss. The technology that we were investigating in the Advanced Technologies group was a kind of advanced version of the sorts of source separation that had been used previously. And they were more sophisticated. They were using deep neural networks, so using machine learning, using AI, in order to be able to adapt frame by frame to the audio and understand where the speech content was, to be able to draw that out, we realized early on in this process. So just to acknowledge, the initial work was carried out by my colleagues, Hadib Mehrabi and Chris Pike. And when looking at these early prototypes, we realized how much potential there was to change the way that we deliver speech enhancement on Sonos products and specifically on the Arc Ultra, which big part of my work was making sure that we were Able to deliver that. That we could do things like adapt to all of the different kinds of scenes that you would encounter within film and television audio and be able to do this with something that was able to run on device in real time in order to deliver an excellent level of speech enhancement, but enable a better sound experience overall and something that people can attune to their own needs. Right. So we've got these four different levels, which is again something new. So with Arc Ultra, we introduced three new levels of enhancement. And with this latest update, you've now got four levels. This is made possible because again, coming back to what we're able to do with the software, we're now able to not just use things like equalization or compression techniques that are commonly used in speech enhancement. We're able to actually pull that speech out and use that capability as a tool within the speech enhancement feature in order to understand where the speech is and use that to really emphasize the speech content according to the particular level that the user chooses.
- [11:26]
Eli Woolery
You mentioned the difficulty in hearing dialogue in modern movies. That certainly resonates with me. I mean, in our family, it's also because our kids are loud, but we often have the captions turned on. And I'm curious if either of you have more insights on why that's become the case. I mean, I've heard things like, well, directorial styles like Christopher Nolan, or back in the 40s and 50s, you could count on an actor like a Cary Grant having this sort of made up mid Atlantic style of speech where they were very crisply enunciating everything, or maybe they had a theatrical background and so they were projecting in that way. But now maybe the acting styles have changed. Are there other technical things that either of you are aware of? For the reasons that we have trouble understanding hearing the dialogue these days, there.
- [12:08]
Matt Benetton
Are all sorts of things that contribute to this. You're absolutely right. There are things about the way that content is created now and the way that actors are directed. You know, lynch and Nolan are people who want things to be gritty, they want things to be real. And when you're creating those environments, one of the things that you might want to do is not to have the dialogue jumping out. Right. You can kind of understand these sorts of decisions. But on top of that, with just the volume of content that's being produced today, there are some smaller studios and stuff where, you know, maybe they don't have as much time to work on dialogue. And as such, you know, there are things that kind of get rushed through, particularly when you're talking about not obviously your big films and stuff like that, but your more casual TV programs. In terms of why people want to use dialogue enhancement, there are all sorts of environmental reasons that dialog might be more difficult to hear. Right. You might have the aircon on. You might have someone doing some noisy DIY next door. Well, as you said, right, your kids could be loud. All sorts of reasons to use it that aren't just based around the content.
- [13:11]
Aaron Walter
Talk to us a little bit about the hearing loss aspect because you had an interesting partnership around hearing loss and accessibility that I found fascinating, especially for a company like Sonos. It was a little unexpected.
- [13:22]
Matt Benetton
Let me start at the beginning of the story, which was, as I said, we had these prototypes of what this speech extraction could do. And at that point, myself and my manager, James Nesfield, we were talking through what this could mean for the future of a speech enhancement feature. And there were people in our personal lives who we felt could really benefit from something that went beyond what we had experienced was commercially available at that time. James's mother and my father in law both struggled with dialogue. My father in law had been using hearing aids for around a year, I think, at that point. And if you know anybody who uses hearing aids, you kind of need to spend a lot of time working with an audiologist to get them tuned properly so that you get the best experience. So when you start using those, you don't really want to have them in when you're watching content. Cause they're probably not gonna be giving you the best kind of sound experience.
- [14:14]
Aaron Walter
And you can get overwhelmed because, you know, if it's an action film, it's like quiet dialogue and all of a sudden it's explosive, overwhelming sound.
- [14:23]
Matt Benetton
Absolutely. And that touches on another really, really important point. So the charity that we worked with was the Royal National Institute for Deaf People here in the uk. They're a leading hearing loss charity. One of the things that we learned from them touches on exactly that point that you mentioned about sort of dynamic range, which is a phenomenon which is called recruit. So loudness recruitment is this kind of awful phenomenon in which not only do quieter sounds become more difficult to hear, but your threshold for louder sounds reduces. Right? So it's compressing that dynamic range of comfortable sound. And as a result of this, you know, we learned that, okay, having more control over the speech will allow us to do more, to kind of compress that range of the overall audio and deliver an experience that allows people to still be immersed, but to be able to hear that dialogue and to be able to engage in it. Learning this stuff from them was really important, right? The RMID taught us a huge amount, but so much of the value came from the participants that the RMID recruited to take part in the design process. So we had several phases of listening tests. We would invite people to come in and trial prototypes as we were developing this in order to understand what people really wanted, what people needed, in order to be able to enjoy content and ultimately to get their feedback on the feature as it ended up.
- [15:50]
Aaron Walter
We'll return to the conversation after this quick break.
- [15:57]
N/A
Clients want It All. A slick looking site that can run their business and scale with their success. Wix Studio is built just for that. Plan out your client's whole site in seconds with AI powered site mapping and wireframing. Then when everyone's on the same page, you can jump into the creative stuff. Starting off in Figma or in WIX Studio Editor with super precise layout tools like Grid Stack and Flexbox. Go above and beyond the brief with no code, animations, custom CSS and built in business solutions and make your whole vision responsive in just a single click. It's super easy and there's zero need to break a sweat when clients grow fast. A dynamic CMS with global design settings and reusable assets lets you turn one page into hundreds, design smoother and deliver sooner. Go to wix studio.com that's wix studio.com.
- [17:01]
Aaron Walter
And now back to the show.
- [17:08]
Eli Woolery
So this is a slight diversion from the current topic, which I think is interesting and we'll probably come back to. But before we lose the thread, I wanted to ask nufoor about this acquisition. So you were on the R and D team at might, you got acquired, you had this technology. It sounds like it was integrated very successfully. I wonder if you have any advice for teams because that happens a lot in technology. A team will get absorbed into a larger organization and it doesn't always work out great. Sometimes the technology they try to integrate, it doesn't happen or something fails. What conditions led to it becoming a successful integration of the team and the technology.
- [17:42]
Nafur Van Appledorn
One of the most important aspects that people within the company which acquires you get a feeling that they understand the technology, make it tangible for them. We had a lot of prototypes, we send out offices all over the world to listen to it, to have demos with it and then also when we started this project we believe really innovation is iteration and we 3D printed almost like a new design every day and we would send out some of the more successful designs to Other teams like to people from the radio team so they could do their electromagnetic measurements, to electrical engineers so they can do their measurements, to the ID people, so they can see does it fit in their design. And I think that's really key to involve all the different teams in this innovation, to make there also an owner of this idea.
- [18:24]
Aaron Walter
One thing that's always been fascinating to me about Sonos as a company is this ethos of engineering and design excellence together. It's a consumer brand, and to have the widest appeal, you need products that are easy to use, that kind of fit into various lifestyles, various homes. There's a lot of variables that y' all are designing for. Could you share with us how design and engineering work together at Sonos? What's that like?
- [18:55]
Nafur Van Appledorn
I mean, design is leading from user experience in that perspective. And if you look at new Arc Ultra, it doesn't have a shape which is optimal for how transducers are shaped. So that's like, one of the biggest challenges. How do you fit speakers inside a very long bar with an almost oval shape? That's the first challenge. It looks very soft into your interior. Not ideal for transducers. That's what's kind of leading. And we are looking like, how do we fit our technology inside that IDE intent? Fun story there is. When we finally reshaped our technology from the MITEC position into what would fit in the Sonos soundbar, ID came to us. We're like, yeah, if you can slightly reduce it by a couple more millimeters, we're blocking even less TVs or almost no TV anymore on the bottom side, because you want to be the soundbar as flat as possible, so you're not covering the bottom side of a TV. I mean, that's the kind of us we're getting. And because we could iterate and 3D print so quickly, we could test with ID like, hey, is that possible with our technology? And it turned out it was that exploration together of what's possible, what can we enable for our users? I think that makes us really strong in delivering those experiences.
- [19:59]
Aaron Walter
And are design and engineers sitting together, working directly together for this sort of thing. When you're iterating through different 3D printed options.
- [20:10]
Nafur Van Appledorn
Yeah, we're really a global company, so we have people in Santa Barbara, Boston, uk, the Netherlands, Asia. One thing we do is send a lot of samples. Nice thing of 3D printing is you can send a 3D file overseas and somebody prints it out and does a fitting test. And we have a lot of get together moments where at key moments we come together. Like hey, we assemble even all those parts together and then you're sitting next to the ID person, next to the electrical engineer, the project manager. That's really essential for that collaboration. What I also found very fascinating is once we have that product, we don't only sit together with engineers, not only with users, but we also sit together with our sound port, led by Charles Martin, with content creators and then we listen to their content on the device. Like, hey, how do you want your content to sound on our product? Doing that loop not only with engineers, not only with users, but also content creators that creates very innovative products.
- [21:02]
Aaron Walter
Are there values or specific design principles that guide your work across the board? Kind of the high level, beacon on the hill type of principles that guide you.
- [21:14]
Nafur Van Appledorn
So Sonos has three guiding principles in mind. So first one is great sound, where we work together with like acoustic and software engineers led by Giles Martin with the soundboard to really match the creator's intent. We have the effortless use, like easy to set up connectivity, really like the Sonos ecosystem and of course the premium design. So they should really fit inside customers. Their lives blend into the furniture. Those three are really the pillars of Sonos design.
- [21:40]
Aaron Walter
So you gave us an example of how a key innovation came about. There was an acquisition, there's this integration process iterating with the design team. Fascinating to hear that you're emailing back and forth, they're sharing files, 3D files, and people are printing in different places, which is a cool solution to a global workforce. What else could you tell us about the R and D process? Finding out, like what are the core challenges or the opportunities that the customer really wants to see from Sonos, like the new products that come out. Is there an R and D team? Is that innovation kind of happening within existing teams?
- [22:18]
Nafur Van Appledorn
Yeah, we have like an innovation team which is dedicated to doing this kind of research. But like innovation can happen anywhere, people can reach out. It's very open. If you have an id, we have the concept of hack weeks which we organize a couple times year where everybody can join in with an id. Innovation is also very chaotic and we divide innovation in two parts. One is we call more invention, which is guided from first principles, technology discoveries and such. And then the other side to it is the exploration which is defined from user experience, like what does customers and users want? And that's kind of an iterative loop where we try to find each other from a technical perspective and from a user perspective. And if that engagement is Fruitful. Then we decide to translate that into a product.
- [23:04]
Eli Woolery
Matt, we found a quote where you're talking about kind of going back into the speech enhancement, that it's not just a practical thing, but there's an emotional aspect to it where a family might be coming together or a group of friends to watch a show. And if one member, let's just say, like in my family, my dad's a bit older and his hearing isn't great. So if he's not able to participate, that kind of ruins experience and maybe talk a little bit more about that emotional side.
- [23:30]
Matt Benetton
Absolutely. There are a couple of different angles to this. Again, something that I've got kind of personal experience with, and then stuff that we learned from the RMID that didn't really come to mind initially. So on the personal front, quite often we'd be watching content with family members who might be struggling with dialogue, and you're always trying to do something. Fortunately, on the Arc Ultra, you know, we had speech announcement, but back when we were developing this, we had the toggle on off speech enhancement on arc, which was good. It did a good job. But there was still content that was too challenging. And that did affect that feeling of being able to listen, be able to engage in content together, to be able to do this as a kind of inclusive experience. And so looking for ways to enable that for people so that they can enjoy content with loved ones is something that we're very happy to have been able to do. And it's one of the key motivating factors behind the R&ID collaboration. I just want to mention quickly that there's an interesting tension that happens when you're working on one of our players with a feature like this, because, as Nafur said, we want to create products that deliver excellent sound experiences. And we knew that with Arc Ultra, we had an excellent sound experience. And that was thanks to working with Giles Martin and the soundboard people like Chris Jenkins, who helped us to tune not just a feature like this, but the product as a whole. And with a feature like this, when we brought the RNID on board, they're looking at, okay, how do we make this clear for people who have this wide array of different types of hearing loss? On the other hand, we had people who wanted to make sure that the product always sounded excellent. And so this tension was really interesting to negotiate. And at the end of the day, what we ended up with is coming back to this point of something that allows people to engage. That quality is there because we've been working with the Soundboard. And that dialogue clarity is there because we've been working with the R&ID that allows for these inclusive listening experiences. The second point I want to mention is something that was brought to my attention by the great R&ID researchers that we worked with, Lauren Ward and Alastair Moore, which is that engaging in content, the social impact of that isn't just about watching with people. It's about not being left out of important moments that are happening. Right, Important cultural moments unfold through the media that we watch, through the television programs, through the news stories. And we're now able to allow more people into that, to engage with that content and allow them to better engage with their communities. And I think that's fantastic message that I didn't anticipate going into this work.
- [26:05]
Aaron Walter
I mentioned earlier that I have a lot of Sonos devices, and the reason why I do is because I love music, and music is just such an important part of my life, and I feel like it's an important part of any creative life. It's so stimulating, fulfilling, healing connects you with other people. What role does music play in each one of your lives?
- [26:25]
Nafur Van Appledorn
When I was younger, I often like sinus infections, so I would hear very badly. Early on, it was hard to hear rhythm, sound, and that kind of thing. So it makes me really happy to hear about this kind of features. But when it got resolved in older age, I started to discover dancing, music and everything and what I love from it. I'm really an engineer by heart, but I can translate all my work to an experience by playing music to somebody like, hey, this is what I'm doing. And they understand that you can talk through this universal language of sound and music. And for me, it's a way of communicating, bringing people together. It's often like those more emotional moments, whether for the better or worse moments in life. And that's what's really defined by music.
- [27:04]
Matt Benetton
That's really interesting that you have a background where you struggled with your hearing. I actually also do. I had chronic ear infections growing up, and so that's one of the motivators behind the speech announcement features. Music is what brought me to technology. I've been a musician for longer than I've been an engineer. I got into music at a young age and I've never looked back. And being able to work somewhere like Sonos, where I can merge these two passions, is hugely important to me. And to know that I'm playing a role in delivering excellent sound experiences to people, you know, helping people enjoy music, helping people enjoy the content that they're watching is awesome. So very passionate about music and love that intersection between creativity and engineering.
- [27:43]
Eli Woolery
What's inspiring each of you right now? And it doesn't have to be a work related thing, could be movie music, it could be a podcast or a book.
- [27:53]
Nafur Van Appledorn
I'm really into like the movies from Veritasium where they explain all kinds of engineering concepts and I'm always thinking like how can I use like something from a total different industry to improve a better sound quality to people. That's what really gives me motivated and inspired.
- [28:07]
Aaron Walter
Veritasium is what you called it?
- [28:08]
Nafur Van Appledorn
Veritasium? Yeah, like on YouTube channel they explain all kind of engineering, very complex engineering solutions and you really feel like, hey, I'm understanding this concept and you start to think about it and use it and have more than often that I was inspired to explore a new field in engineering.
- [28:25]
Aaron Walter
I watched this guy.
- [28:25]
Nafur Van Appledorn
He's awesome, right? Yeah.
- [28:27]
Matt Benetton
Yeah. For a long time I've been a fan of the first Dune book by Frank Herbert and over the last year or so I've been reading the follow ons to that and I've been really getting into some of the concepts. Some of this stuff is absolutely wild. These sort of insights into this kind of crazy idea of prescience and genetic memory and stuff like that. And that's something that I just found sort of really interesting and kind of I find turning over in my mind every so often. The imagination of Frank Herbert is really astounding. Love those books.
- [28:59]
Aaron Walter
Well, this is great. The work that you're doing is interesting. I love that you're bringing accessibility to sound, but also in the process of making sound more accessible to more people, you're also making it just richer for everyone. So I love that. Matt Benetton and Nafor Van Appledoorn, thank you so much for being on Design Better.
- [29:18]
Matt Benetton
Thanks so much for having us.
- [29:20]
Nafur Van Appledorn
Thank you for having us.
- [29:25]
Aaron Walter
This episode was produced by Eli Woolery and me, Aaron Walter, with engineering and production support from Brian Paik of Pacific Audio. If you found this episode useful, we hope that you'll leave us a review on Apple Podcasts, Spotify or wherever you listen to finer shows or simply drop a link to the show in your team slack channel designbetterpodcast.com it'll really help others discover the show. Until next time.
- [29:55]
Nafur Van Appledorn
Sam.