AssemblyAI CEO Dylan Fox on 120 Million Voice Conversations a Week

Summary

---

Sourcery Podcast Summary

Episode: AssemblyAI CEO Dylan Fox on 120 Million Voice Conversations a Week

Date: July 31, 2026
Podcast: Sourcery with Molly O'Shea
Guest: Dylan Fox, CEO of AssemblyAI

---

Episode Overview

In this episode, Molly O’Shea interviews Dylan Fox, CEO of AssemblyAI, a leading voice AI infrastructure provider processing over 120 million weekly conversations. The discussion dives into the explosive growth in voice AI, the major technology and market trends fueling this shift, differentiation from competitors, how voice is transforming industries, how AssemblyAI builds and trains models at scale, and what the future holds for voice as a new interface. They also explore practical and philosophical questions around voice agents, model bias, language, data, and balancing user experience as AI voice agents become ubiquitous.

---

Key Discussion Points & Insights

AssemblyAI’s Scale and Industry Explosion

• Massive growth:

  • AssemblyAI’s weekly usage is up 800% over three years, now processing over 120 million weekly conversations and 2 million hours of voice.
  • This is “a little over 4x the amount of daily volume that’s going to YouTube” ([00:00], [02:08]-[03:20]).
  • “About a million developers, a little over a million … on the platform now. And, you know, 40% of those … signed up to the API last year.” ([02:54])
  • Huge range of use cases: drive-thru ordering, healthcare, note-taking, field service, toys, and more.

• Infrastructure focus:

  • AssemblyAI provides foundational voice AI models and infrastructure, not end-user applications.
  • “We are 100% focused on voice AI infrastructure. So we don’t do anything at the application layer… Voice models are amazing for all those use cases…” ([10:26])
  • “We’re like the AWS for voice capabilities. That’s the product vision.” ([36:20])

Macro Trends Driving Voice AI Adoption

• Technological gains:

  • Models are “getting more accurate, they’re getting faster, more controllable, more capabilities” ([00:18], [06:42]).
  • Context-aware AI: AssemblyAI’s newest models can intake and act on environmental context (e.g., discerning a McDonald's order vs. background noise).
    • “With our models, you can give them context: ‘Hey, you’re taking a McDonald’s order, focus on the person ordering, ignore the kids screaming…’” ([00:34], [06:54])

• Explosion of ecosystem/infrastructure:

  • Companies building on stacks including reasoning/vision/vector DBs, enabling sophisticated applications beyond just the voice layer.

• Rise of coding agents/democratized development:

  • Coding agents and agentic infrastructure mean “everyone’s an engineer”—even non-tech SMBs are automating their back office with code that interacts directly with AssemblyAI’s infrastructure.
  • “Our TAM has just increased by 100x … now it’s like, anyone.” ([01:01], [09:23])
  • “We saw this small business, like lawn care chain, sign up … small businesses are automating parts of their back office using Lovable or Cloud Code...” ([09:35])

Differentiation from Competitors

• “We don’t build end-user products. Our focus is multi-layered: model research, scalable inference (handling 4x YouTube volume!), orchestration, and developer/agent experience.” ([10:26])
• Efficiency comes from “years of engineering work, people like Ben, amazing researchers, deep customer proximity—not just building models, but continually refining based on specific customer contexts and error cases.” ([12:47])
• “Most AI note takers are using Assembly as the voice infrastructure … our models are the most scalable in the market…” ([11:51])

Building an AI-Native Company (Internal Use of Agents)

• “Everything we do is AI native … I have my own agent, the ‘Dylan Claw,’ which handles meeting notes, creates slides, merges presentations, automates onboarding, even submits PRs to GitHub…” ([15:14])
• “We’re about 80 people, huge scale for company size … everyone uses AI agents … even rebuilt our website with Claude and Vercel, now anyone can make changes via Slack.” ([15:38], [17:03])

Sovereignty, Data Sensitivity & Local Models

• “You can deploy our platform on-premise, fully self-hosted and private… big uptick for sensitive, sovereign needs.” ([17:18])
• “For voice, open source still lags far behind—many who fine-tune open models end up returning to Assembly due to quality/maintenance.” ([17:41])

Future of Voice as an Interface

• Voice as new “dimension” and primary data capture:

  • “Voice is a reliable form of data capture now … you can actually talk to your computer, have it listen, take notes, update charts in healthcare, or help field service techs.” ([18:46])
  • “Touchscreens didn’t replace keyboards … voice is another dimension—I want to type and talk to my computer.” ([19:57])
  • “Consumer electronics—TVs, toys, IoT—that can be controlled reliably by voice are coming fast.” ([18:46], [21:11])

• Humanoid robots and agentic confusion:

  • “Robots need to distinguish among multiple speakers—still a hard unsolved problem. Our models are making progress but background voices are tricky.” ([22:33])
  • “In five years kids are just going to talk at things, expecting them to understand…” ([24:37])

Language Model Challenges

• Key insights:
  • “It’s less about the science, more about the culture … policy alignment for a native speaker.” ([25:21])
  • Localization matters: “Local [vendors] typically are the best … they speak the language, quickly identify data quality…” ([26:32])
  • Models must handle over 20 languages natively and switch in real-time without direction ([25:55])

Model Training at Scale

• “Probably 75% of it is the data you’re training on. Data mixtures, frequent model updates, constant feedback loops—model quality is always improving on our platform.” ([31:40])
• “Some apps want all background voices; others (like McDonald’s) want only the main speaker.” ([32:49])

---

Notable Quotes & Memorable Moments

• On product market fit and tech leaps:

  • “I always viewed this journey as similar to self-driving cars, where it wasn’t a question of ‘is the product market fit going to come?’ It was like, no, the technology just sucks. And as it gets better and better, you’re going to continuously cross these thresholds…”
    — Dylan Fox ([05:38])

• On coding agent transformation:

  • “Now everyone’s an engineer because they have coding agents ... Our TAM has just increased by 100x.”
    — Dylan Fox ([09:23])

• On voice as a new interface:

  • “Voice is a reliable form of data capture now for the first time probably ever… you can actually talk to your computer now.”
    — Dylan Fox ([18:46])

• On model training realities:

  • “For any AI model, there’s always step functions in algorithms and architectures, but the data is just so important… We spend a huge amount of time just on data.”
    — Dylan Fox ([31:40])

• On the future of human-computer interaction:

  • “It’s easier to predict 24, 36 months than the next 12… I think over the next year we’ll see a lot more consumer applications … toys, games, consumer electronics … voice as a core dimension.”
    — Dylan Fox ([36:57])

• On the challenge of agent/human transparency:

  • “Every text agent I talk to discloses it’s an AI… Voice is different. The goal [now] is to try to trick the human into believing that it’s also a human ... There’s something we still have to figure out about the UX.”
    — Dylan Fox ([36:57], [39:42])

---

Timestamps for Key Segments

• [00:00] AssemblyAI usage stats (weekly conversations, API calls, developer adoption)
• [06:42] Three macro trends in voice AI: model improvements, broader ecosystem, coding agents
• [10:26] Differentiation from competitors (infrastructure vs. applications)
• [12:47] Achieving efficiency and scale internally
• [15:14] Running an AI-native company—practical internal use of agents
• [17:18] Data sensitivity, sovereignty, on-prem deployment
• [18:46] Voice as data capture, interface paradigm shift
• [22:33] Challenges in humanoid robotics and speech disambiguation
• [25:21] Model expansion into new languages, policy alignment, localization
• [31:40] How model training and iteration really works at scale
• [34:07] Team composition
• [36:57] Forward-looking excitement, product vision, and AI UX paradox
• [39:42] Dilemma: should voice agents pretend to be human or not?
• [43:25] Personal circle of inspiration and support

---

Final Reflections and Outlook

• Emphasis on continual product and model improvement, customer feedback, and the centrality of voice in future computing.
• The democratization and ubiquitous integration of advanced, context-aware voice AI is accelerating, reshaping consumer interfaces, and enabling entirely new classes of products and services.
• Key future milestone: widespread adoption of voice as a core interface layer in consumer electronics—and resolving the tricky questions of transparency and trust with AI agents.

---

“In five years, kids are just going to talk at things, expecting them to understand them. And that’s the shift with computing in general that we’re going to see.”
— Dylan Fox, [24:37]

---

Transcript

[Dylan Fox]

The amount of weekly conversations that assembly handles through our APIs every week is up over 800% over the last three years. On a peak week, there'll be something like over 120 million voice conversations going through our platform. Over 2 million hours of voice. It's a little over 4x the amount of daily volume that's going to YouTube. So they're getting more accurate, they're getting faster, more controllable, more capabilities. The newest voice models that we just launched, you can give them context about the environment they're operating in. If you drive to a McDonald's and you order with your voice, that voice AI system has no clue clue that it's taking a McDonald's order. But with our models, you can give them context on, hey, you're taking a McDonald's order. You want to focus on the person that's ordering. Ignore the kids that are screaming in the background. And so we're seeing this huge inflection now of, like, you have enterprises, small businesses that can build with API infrastructure. Now our TAM has just increased by 100x because we're not just selling to engineering teams within product companies. It's now like, anyone.

[Host of Sorcery Podcast]

Dylan Fox, welcome to sorcery.

[Dylan Fox]

Yeah, thanks for having me here.

[Host of Sorcery Podcast]

So we're gonna have a very fun conversation on voice. We were just gabbing off camera about how much I love voice and how I've used it to start this podcast and do all these interviews specifically, like, use the transcripts that I capture from recordings into different kinds of formats. I started it for investing and so I would take that, I would repackage it, I would make like open source expert calls and open source memos and all this kind of stuff just from capturing that data. Now you have built the infrastructure for how that even works. So I want to get into that. I want to get into everything from the business model, how you've built out the products. You've been doing this for a very long time and there's a lot of new entrants to the category. So it'd be great to learn what is BS and what's not.

[Dylan Fox]

Yeah, yeah.

[Host of Sorcery Podcast]

As well as, like, areas that you're most excited about. But before we start, I guess it would be great to kind of capture and illustrate just how big assembly AI has gotten, how much data you train on.

[Dylan Fox]

Yeah. So the, the demand for voice applications and the applications that companies are building on our infrastructure is over the last two years in particular, really started to explode. One stat I was just looking at before I came over here was like, the amount of weekly conversations that assembly handles through our APIs every week is up over 800% over the last three years. Oh, my God. So now, you know, on a given week, on a peak week, there'll be something like over 120 million conversations, voice conversations going through our platform. Over 2 million hours of voice, which as of December of this, of this past year was 4x or a little over 4x. The amount of, of daily volume that's going to YouTube, which when I found out was like, wait, I need to double check this, because, you know, that can't be right. But yeah, the volume is huge. There's almost 100 million API calls a day coming against our API. About a million developers, a little over a million developers on the platform now. And, you know, 40% of those developers signed up to the API last year. And so while we started this a long time ago, it's really over the last, like two, three years, where voice and edits accelerating, inflecting voice is becoming just a core part of software and increasingly hardware too. And so we're seeing that through our platform because we're the infrastructure really under, like, under all of it.

[Host of Sorcery Podcast]

So if you look back over time, you guys went through yc.

[Dylan Fox]

Yep.

[Host of Sorcery Podcast]

Did you think that this would be the reason why Voice would take off and people talking to their phones and talking to their computers and that kind of thing? Like, what did you think?

[Dylan Fox]

Yeah, I mean, I did, which is why, you know, I've spent so much time on this, because we were actually the very first AI batch in YC when we went through yc. And so it was Daniel Gross, who, if you know of Daniel now at Meta, he started the AI batch at YC back when we went through in 2017. And it was me and like five other companies, and we got $100,000 in GPU credits. That was like our perk. Which now seems like, cute, right? It's like, wow, that's nothing. But back then it was like, wow, $100,000 of, like, Nvidia K80 usage. This is amazing. But the AI ecosystem back then was just in its infancy. I was going to the very first TensorFlow meetups. Just to put it into perspective. No one was really using AI in production yet. But for me personally, I had gotten an Amazon Echo and was really into voice interfaces and talking to this hardware. That worked well. Because my experience with Voice prior had been everything was terrible. And so then I went looking to find APIs that I could just in my spare time back then, build with and I couldn't find anything and all the technology kind of sucked at the time, but I felt like, okay, over the next 10 years this is going to get so much better. And when it does, if it's easily accessible to developers like a Twilio or Stripe, you're going to have so many creative people build really cool things. And so now, you know, that's what gets us excited when we see, you know, our customers like granola, right? Granola. Building these like amazing products that are helping companies and professionals like run their businesses. You know, we have customers like Tolans that are building, we were talking about it before, like AI companions that hundreds of thousands of people are talking to for like hours every day in the healthcare space. Athelis Kamir, it's a big health tech company building AI scribes for doctors and it's really cool to see what people are building. So I think that, to go back to your question, I always viewed this journey as similar to self driving cars, where it wasn't a question of, oh, is the product market fit going to come? It was like, no, the technology just sucks. And as it gets better and better, you're going to continuously cross these thresholds where you open up more tranches of the market. And that's still happening. And there's new macro trends that are happening that are accelerating, accelerating that further beyond just the like core technology getting better. I can talk about those, but that was very much like the like thesis behind the company, right? Like for me it was like, this technology is going to get better. It's going to enable like so many new applications and voice to be something you can really build with and like a core, a core like modality across, across like in our lives. And it just felt like something really cool to work on.

[Host of Sorcery Podcast]

So what are those core macro trends?

[Dylan Fox]

So there's really three things that we see that are causing everything. All our metrics, like inflect. The first, which is what we control is voice models are getting just much better. So they're getting more accurate, they're getting faster, more controllable, more capabilities. The newest voice models that we just launched, for example, you can give them context about the environment they're operating in and give you an example of that is like today if you drive through a McDonald's and you order with your voice and there's a voice AI system there, that voice AI system has no clue that it's taking a McDonald's order, right? It has no context. It's like fresh. But with our models you can give Them context on, hey, you're taking a McDonald's order, you want to focus on the person that's ordering, ignore the kids that are screaming in the background. And those types of capabilities make them much more accurate. And then they're also faster and they're lower cost so they can be more widely deployed. So you have these voice models that are just getting better across these key dimensions. That's like what we control. But the other two macro trends, the one is number two in this list of three is like you have this ecosystem of AI infrastructure getting built out. So you have these reasoning models, you have vector databases, you have other AI models across other modalities. And so now when you're a company wanting to build something like drive through voice ordering or AI note taking or healthcare, you don't just have the voice models, you have the rest of the infrastructure. You need to go build the thing you're trying to build. And that has, you know, been accelerating over the last couple years. And then the third, which is new, is coding agents. So we're now seeing companies sign up spending a lot of money on API that like not even two years ago, like 18 months ago, like I would never think of them as our target customers because we're an API platform. So we, you would think like, oh, you're selling to engineering teams, product teams, but now everyone's an engineer because they have coding agents, right? And so we saw this small business like lawn care chain sign up and we're like, what do they do with the API? And a lot of these small businesses are automating parts of their back office using lovable or cloud code or cursor or whatever, replit. And those coding agents are building on our infrastructure. And so we're seeing this huge inflection now of like you have enterprises, you have small businesses that can build with API infrastructure now. So I think about this as like our TAM has just increased by 100x because we're not just selling to engineering teams within product companies. It's now like anyone. And we're seeing some global enterprises now come in and build their own software for a bunch of different use cases. And they're doing that on top of our infrastructure. And when I go talk to them, like, why are they doing this? It's like, well, they have two people working on this with a bunch of coding agents. And so those three things, it's the voice models are getting better, like the broader ecosystem of AI technology. And then the third is anyone can build this stuff now because of coding agents. Are really causing voice to be way more widely deployed and across just like so many different use cases.

[Host of Sorcery Podcast]

So for people that are new to this space, it would be helpful to explain your differentiation between the likes of like 11 labs. Obviously Sierra is in a different bucket too. But like how do you fit in within that world?

[Dylan Fox]

Yeah. So we are 100% focused on voice AI infrastructure. So we don't do anything at the application layer. Where we focus is the models. So we create models that are. Voice models are amazing for all those use cases I spoke about. So healthcare, we have medical focus models, drive through voice ordering, contact center, AI note taking, we then build out the inference around those models. So to handle 4x the amount of YouTube volume on a day, 120 million conversations a week and growing over 100% year over year, there's a lot of infrastructure we have to build out to make sure everything scales is available, is like super fast, is low cost for customers. So we build out a ton of infrastructure on inference around our models. And then we also do a lot around the orchestration layer. So if you want to build a voice agent, if you want to understand speakers, if you want to translate data, we have a ton of software at the orchestration layer that companies can leverage. And then above that we have this amazing developer experience, agentic coding experience. So agents can easily build with our stuff. And when we say infrastructure, I think a lot of times people think like, oh, you're just creating the model weights, you're just creating models. That's a part of it. But those other parts are equally important to us. The inference, the orchestration, the developer and agent experience. And so we're 100% focused on that and across all the different use cases that we focus on and making sure everything's tailored for those. And so the reason most AI note takers are using assembly as the voice infrastructure is because our models are the most scalable. Right. If you have tons of people, millions of users that you're doing live note taking for, we have the most scalable models in the market that are super low cost, high accuracy, very fast, can burst to huge amounts of traffic, you don't have to stand up servers as a customer. So all that is the area that we're focused on and we're enabling companies to build on top of that.

[Host of Sorcery Podcast]

How did you get it to become so efficient?

[Dylan Fox]

You know, years of engineering work, People like Ben, at our company, we have an amazing team of engineers, of researchers that just operate so closely to customers that they really understand like how this stuff is being deployed. I think that's the biggest difference between us and like a lab at a bigger company. So. So we have a lot of researchers, research engineers that will come to assembly from a bigger company and they're so far removed from the customer. So they just create their model, they benchmark it and then it's like okay, I'm done. But you have to know, all right, who are the customers? How are they using this? What do they care about what errors break their application and what errors don't matter. And then how do you optimize both your models and your infrastructure for that? So we've just spent years and years building out the infrastructure to make this stuff work. So infrastructure that's like cross region and cross cloud and stuff that just will really scale. But it's probably like half our work is spent just making our infrastructure more and more scalable.

[Sponsor/Advertisement Voice]

This episode is brought to you by Brex. My favorite. You become what you spend on and I refuse to spend my time on

[Host of Sorcery Podcast]

work that shouldn't exist.

[Sponsor/Advertisement Voice]

Expense reports, receipt chasing and manual closes the companies building. What's next? From Vercel, OpenAI, Anthropic Granola and Deepgram all made the same call. They all run on Brex. Brex is the intelligent finance platform that combines cards, expenses and banking into a single stack. With agentic finance built in AI agents that handle expenses automatically enforce policy before spend happens and close your books in minutes. That's why Sorcery runs on Brex. So I can spend time on building and not busy work. It's time to get Rex AF. Learn more at brex.com sorcery that's B-R-E-X.com S O U R C E R-Y bye.

[Host of Sorcery Podcast]

I feel like between like well the SaaS apocalypse is one thing, but even leading up to this point with AI helping companies restructure, whether it's on the kind of internal side with developers or on the outward facing side in marketing, that kind of thing. I'd be really interested to hear how you've re architectured the company internally to do that.

[Dylan Fox]

Yeah, it's a great question. I mean we're AI native company. Like everything we do is AI native. And so what I mean by that is like I have even me personally, I have my own agent that I've built, I call it my Dylan Claw on my computer and it has my meeting notes and it can make slides for me and it can do everything. And so I just onboarded someone new to the company and I was going through My slide deck and it's like, oh, I was talking through a part of the deck that was a bit clunky and then because my agent has access to the slides and can make new slides and was, you know, has the notes from that meeting powered by assembly, I was like, hey, go create a new version of the deck that is a bit easier to walk through and like look at the, the transcript from that most recent onboarding to make some changes and I want to like merge these two decks together and then like 20 minutes later it's done and it's perfect and I'll have to edit it. We have a agent that has access to all the company information so our metrics and the whole company uses it. It can submit PRS in GitHub across our products. And so I think like, you know, we've always stayed very small. We're only, I think we'll be 80 people at the company pretty soon. So we have huge scale for the size of company that we're at and we try to be intentional about that. But I think for us we've just really leaned in to try to use AI across everything. Like the most recent example is we had Claude rebuild our whole website off of webflow and just deploy it on Vercel. And now anyone, people in Slack can just make changes to the website. So you get off a customer call, you're like, oh, this part needs updating and it's like boom, you can just change it. So it's things like that that have I think helped us leverage AI across the company.

[Host of Sorcery Podcast]

Sovereign AI has been a huge topic and you mentioned some of your customers that are in categories that are a bit more critical and sensitive for data. But we are seeing this across the board because you can get copied by the labs.

[Dylan Fox]

I mean, yeah, so you can deploy our platform, you know, on premise, you can run it all self hosted and that's totally private and we do see an uptick in that for sure. I think that you know, it's this mix like invoice is so early where you know, we, we the models and the capabilities are getting better so quickly that it's still hard to like to branch out. You know what I mean by that? Like, like we've seen some customers maybe six months ago or a year ago, they'll take an open source model and fine tune it or something and then you know, they're now like, okay, this is like shit, this is like really behind and it's like nightmare to maintain and I don't want to work on this and so they'll come to assembly. And I think voice is much earlier where we haven't seen as much of that as you're probably talking about in like yeah, if you're a bank and everything's going into cloud or something, it's like, wait, all my proprietary kind of knowledge is going out. But we are working on ways for companies to create like their own custom models, deploy them on our inference or run them on their own cloud. But it's earlier there for voice for sure.

[Host of Sorcery Podcast]

I was thinking about it because when we were at the raise summit, this was a hot take that. Max Cook, who's a sector head at CO2, said he thinks the keyboard and mouse is over. We've been through so many different form factors of computers and the human like machine interface, that's like, no, we gotta close the books on the computer keyboard and mouse.

[Dylan Fox]

It's funny when you go into offices and it has like the gooseneck microphones and they're like whispering over. I think you're gonna have. So here's like what I genuinely see. Like voice is a, I would think of the word like modality or interface, but I don't even think those capture, like voice is a reliable form of, let's just call it like data capture now for the first time probably ever in the past year, it's like a reliable form of data capture. And so what does that mean? That means like you can actually talk to your computer now. You can actually have it listen in. You have your computer listen in on your meetings and take notes for you. Doctors can use it during your patient encounter to update the charts. We have a startup, it's called Ciro AI. I don't know if you've heard of them, they're building on the platform, they're creating an AI for field service tech. So if you're a plumber, H VAC technician, they have an app you can run and it will listen in on your service visit and then give you feedback after for like sales coaching. And it's helping those field tech salespeople be so much more successful because like they're actually getting feedback and they never were before. So earning like 20% more take home pay or something. You know, we're seeing it with like toys and with hardware where you can like talk to toys now, which is cool for kids. You know, me as a parent seeing that stuff is really cool. So voice is this, is this reliable form of data capture. And I think when people talk about voice AI, a lot of times they're talking about, like, AI voices, right? And that's a part of it. That's a big part of it. Like, you have voices that are. Sound great and they're amazing. They're so much better. But the other part is like, can the computer, the machine understand you? And that voice capture part, is that reliable? Is that solid? Does that work at scale? But that is the case now. And so I actually think you're going to see, like, touchscreens didn't replace keyboards. It's like you have both, you have the, you know, on a coffee machine, like, there's the rotary dial, like, and there's the button, the touchscreen menu. And I think voice is going to be another dimension. That's probably the way, the way I think about it, voice is going to be a new dimension, but it's going to be an additional dimension. And so for me, like, I like to type, but I also like to talk to my computer. I like to do both things. And I think for, you know, where I get excited is like, consumer electronics that you're going to be able to talk to. So, like your TV, like, it's still crazy that, like most TVs, you can't just be like, hey, put on, like, open up Apple TV and put on, you know, this, this episode of whatever.

[Host of Sorcery Podcast]

They are always listening. I don't know if you saw the succession.

[Dylan Fox]

No, no, no, I didn't see that. No.

[Host of Sorcery Podcast]

Always listening.

[Dylan Fox]

Yeah, but like, it's crazy. That doesn't work yet. And so I don't think that means that computers are just going to be like a screen and you just talk to it because actually we'd get tired of talking, but. But I think it's going to be a dimension that' and so that over the next couple years will happen. And I think that's really cool because I think that will allow for us to be freed of our addiction to screens. Right? Because your input to most computers now is like this touchscreen, this smaller and smaller touchscreen that just captures you, but with voice, that enables more passive hardware and more passive computing experiences where it's like you can talk to things and you can be walking around and you can be free from being a prisoner to the device. So that's what I think is really cool. And that's for sure coming. We have some big consumer electronics companies actively engaged with us, implementing our technology right now for things like that, which is really cool.

[Host of Sorcery Podcast]

What about humanoid robotics?

[Dylan Fox]

Those two. I think that, you know, every time I think about, like the Neo or the figure, I Think about like my. I have a four year old and a three year old and I think about like one of those in the house with them because we have a matic robot and they like love to mess with the matic robot. They broke one because they always go run over to like turn it on because they've realized like you can press the button and turn it on and then they like sit on it and they mess with it. They try to like ride it around. And yeah, I don't know what the weight is of a Neo or something, but yeah, I think you'll have humanoid robots. Obviously the way you're going to want to interact with the humanoid robot is by talking to it. But this is where what we do is so important is because I'll tell you one of the main problems the humanoid robots face today because a lot of them are using our APIs. If you have three people standing next to the robot, it doesn't know who to listen to and it has a hard time disambiguating who's saying what. And so it all just gets kind of jumbled and merged. This is a big problem.

[Host of Sorcery Podcast]

This is using listening data, not video data.

[Dylan Fox]

Yeah, exactly. And even if it's video, like if you're not facing it, maybe it can't see. Right. So like it does have the cameras. But you know, you really need to be able to, like, for you, like if I, if, for me, like if I close my eyes, I can still tell, you know, who's saying what. Right. But for voice agents, even if they're over the phone, this is a big problem with them today. You know, I, last week or last month like called a voice agent and I called a restaurant. It was a voice agent that answered. So it was like an AI that I'm talking to. And I know it's an AI, even though it tries to trick me because I build this stuff. But my son's in the back and he's talking and it's like tripping up the voice agent because it doesn't know that that's a background speaker. It just hears speech. And it's like, I need to respond to this. And those problems are still not solved yet, these voice models. And that's going back to the sovereign AI stuff. I think right now we're still in the phase of like, I need this stuff to work and I just need to like, because there's such an opportunity, if you can be the first, to get it to work. And we're still in that, like, get this stuff to work phase, probably that will be the next 18 months. And yeah, I think that like, yeah, across humanoid robots, consumer electronics, like, you're going to see voice as this dimension that you just like, expect. And the example I think about is, you know, when you see pictures of like kids trying to swipe on TVs or something, because they just expect everything as a touchscreen. I think it's going to be similar where in five years kids are just going to talk at things, expecting them to be able to understand them. And that's the shift with computing in general that we're going to see.

[Host of Sorcery Podcast]

What has it been like to expand the models into translating different languages?

[Dylan Fox]

It's a hard problem. And it's less about. It's less about like the science of it and it's actually more about like the culture of it. Right. So like, our head of research is from, is Japanese, from Japan. And there's like a lot of, you know, there's, there's a lot of like, opinions about, like, how, you know, like how you like, think about it as like, policy alignment. Policy alignment for how you want it to show up for a native speaker. My wife is Danish and so I always have her test our Danish models. And I'm like, is it good? Try it. And she's like, yeah, but this you wouldn't really do. And it's like these details, but they're important details. And so I think it's less about the science and it's more about can you get those details right. And more and more you want to have a single model that can do all these different languages, right? And so our latest model, for example, it's a single model. It can do 20 different languages. You don't have to tell it, it just will figure it out. It can switch across those languages in real time. But if you fix this one thing, then you might break this other thing. And so it's that equilibrium that you're trying to make in these models and you have to have this kind of deep understanding of each language. And it's why actually if you look at local voice AI vendors in certain countries, they actually are typically the best because it's, you know, they speak the language, they understand it and they can more quickly identify, like, which data is good and stuff to train on. So that, that's a big part of it is like that, like policy alignment and like the language expertise per language that you really need.

[Host of Sorcery Podcast]

Would you acquire those companies? Like, how would you get that?

[Dylan Fox]

Yeah, I mean, a big, A big part is like you know, we, we try to build a team of those experts. We have a lot of customers that help out and service those experts and give us that feedback. We partner really closely with customers. A lot of them are not working in sensitive applications, so they opt in to letting us use their data to improve models and their feedback, which is great. So it's a mix of all those things, but that's really where there's still a lot of opportunity to differentiate at these models. And I think for voice, a lot of people will think like, oh voice. Like what people have been telling me forever is like, oh voices. You know, isn't that solved? Like, isn't that just like commoditized technology? But like it's definitely not because there's, there's so many gaps across languages, across different, different verticals and applications and domains that there's big rooms for improvement in.

[Sponsor/Advertisement Voice]

Still, if you're building what's next in AI, you need to know MongoDB, the database platform developers love and built for the agents you're running. MongoDB stores searches and reasons over your data in real time with vector search and embeddings from Voyage AI all in the same system. No separate pipelines, no stitching together 10 different tools. It's why 75% of the Fortune 100 and leading AI native startups run on MongoDB. Build and scale from your first user to billions of vectors. Go to mongodb.com AI to learn more. That's mongodb.com AI to learn. Bye. Assembly AI is a voice AI infrastructure layer millions of developers build on. They build the industry's best speech to text, voice agent and speech understanding models that serve as critical infrastructure for companies like Granola, Heygen Ashby and ClickUp. Their speech to text models lead the industry in accuracy and quality. And their speech understanding models help you go beyond transcription by uncovering insights, identifying speakers and highlighting key information from voice data. You can get started [e-mail] sorcery and get $50 of free credits to start building Voice AI products. That's AssemblyAI.com S-O U R C E

[Host of Sorcery Podcast]

R Y I have a hard question.

[Dylan Fox]

Yeah?

[Host of Sorcery Podcast]

How far away? I'm going to try to deliver this without smiling. How far away are we from translating dogs and cats?

[Dylan Fox]

Dogs and cats? Yeah. I don't think voice will be the interface I think it's going to be. I think it's going to be more like MRI scans or something like sub vocal. That's the thing I really think about too. Will we talk about computers. I think probably the ultimate interface is it's just reading your mind. Right? That's true. You just plug in some USB stick and put it on your head and then it just reads your mind and then it's just like unlimited bandwidth to the machine. So that's probably the solution for the dogs and cats.

[Host of Sorcery Podcast]

I don't know. We both said. We both watched Project Hail Mary and he did translate an alien.

[Dylan Fox]

Yep. So I know, Yeah, I was telling you this before, but, you know, as I was watching that part, I was like, this is way too easy for him to train this model. Basically. Like, he would need, like a lot more data. Uh, but yeah, so where.

[Host of Sorcery Podcast]

Where did he go wrong with that?

[Dylan Fox]

With.

[Host of Sorcery Podcast]

With Rocky?

[Dylan Fox]

With. With Rocky? Well, he solved it. I mean, he. Yeah, they were able to talk. Uh, but yeah, he did. They. They did turn Rocky into more of like a. As we were talking about, like a dog. A dog that I think, like a very intelligent space faring entity.

[Host of Sorcery Podcast]

Yeah, that was a weird realization because I was like, thinking about it. I was like, they both left their planets. I mean, Rocky's supposed to be like 100 or 200 years old.

[Dylan Fox]

Yeah.

[Host of Sorcery Podcast]

Right. And like, he's an adult and he has a lot of responsibilities. But, like, I mean, both of the characters were quite playful and I think that's why the movie was so great.

[Dylan Fox]

Yeah.

[Host of Sorcery Podcast]

But it was definitely super weird.

[Dylan Fox]

Yeah.

[Host of Sorcery Podcast]

Why was Rocky the dog?

[Dylan Fox]

Yeah, they made him like, like, like Grace was like the, the like parrot. But it's probably good for, for viewers

[Host of Sorcery Podcast]

now I'm like, totally off tangent. Okay. So I guess, like, to get deeper into how you train based off of the hundreds of millions of hours of data that you have. How does that really work?

[Dylan Fox]

Yeah, my experience has been it's really probably like 75% of it is like the data that you're training on. Like, I would say for any AI model, it's like there's always these step functions and algorithms and architectures and stuff, but the data is just so important. And so we spent a huge amount of time just on data, trying out different data mixtures, training different models. And we release model updates every couple of weeks. And that's the big benefit that our customers have when they're building on our platform is it's literally constantly getting better. It's not like every six months or every year there's an update. It's like there are constant improvements going out. We have ton of different model versions. We have a ton of different APIs, which we just launched, like a Different API today for different use cases for dictation and like push to talk type use cases. So we're constantly innovating on the model training part. So much of it's about the data. Is the quality good? Is it aligned with like what users want? And I'll just give you an example of that for the speech to text task. Some applications don't want to pick up the background speakers. Some do. So we have some customers who are looking at police body cam footage for legal discovery. They're running that through our platform and they want to know everything that was said by every single speaker. Leave nothing out. But if you're McDonald's taking a voice order, you don't care about anything other than the person in the front seat placing that order. If the kid in the back screams screens like an extra large frosty, it's like you want to be smart enough to ignore that. So there's all these different expectations of these models and that's where I think the alignment of data to the use cases is really the process knowledge that is required to build really good technology. And it's why if you go online, you look at some of these open source benchmarks, it's like, oh, wow. These models are performing similarly, but it's very easy to optimize for like a public open source benchmark. It's very hard to optimize across all these real world applications. And so we spend a ton of time on evals. We have like a whole team internally that works on evals, constantly evaluating our models across like a million different metrics, million different data sets across all these languages. And it's so much of it's about the data.

[Host of Sorcery Podcast]

How is the team constructed? How many employees do you have?

[Dylan Fox]

So we're, yeah, we're almost 80 people at the company. It's pretty much like almost entirely like product research and engineering and for deployed engineers that are just like working really closely with customers.

[Host of Sorcery Podcast]

We didn't really go into your background too much.

[Dylan Fox]

Sure, yeah, yeah, yeah. I mean it's interesting. I sometimes I'm like, wow, I've been doing this a long time. I like really where it goes back to is I taught myself how to program in college and actually like way before that as a kid my brother would build computers like in the, in our basement and I would be on like, like irc. You know of irc? Like totally. Yeah. No, these like I would be playing these like video games and in these like chat rooms with like random people and like building, you know, websites for these like Teams we make. And so I was like always on computers and then kind of got into programming and taught myself how to code in college. And I loved the idea of being able to build something and these, like, these hard problems in computers. So that got me into natural language processing and machine learning. And that's where I really got interested in this natural language understanding, voice area. And it just turned out to be this really cool emerging space where we get to work with all these cool, innovative companies and build new technology. And so for me, that's the really cool thing, building this. I think every company has a DNA and like a reason why they start it or why it started. And for me it was like, I love developer tools. Like, I'm developer by background. Like, that's kind of always what I was passionate about. And giving, like other people tools to build on and seeing like what they can do with it is so fun and cool. And every time we see, you know, some startup builds something on the API and then they like raise the Series A and stuff, it's like, really cool to see. At some point I want to try to figure out, like, what's the cumulative total funds raised of like, all the startups on assembly because just to show like, how much demand is going towards all this stuff. But yeah, my background's really about building things.

[Host of Sorcery Podcast]

So that's why your infrastructure layer versus application.

[Dylan Fox]

Yeah, exactly. I think, you know, companies have their focuses and their, and their DNA and for us it's like infrastructure. And we're really good at scaling stuff and we're really good at creating technologies, technology. And that's our DNA. And that's ultimately like, what our customers are buying from us. It's like, hey, we're this amazing infrastructure platform. We're like the AWS for voice capabilities. That's the product vision that we have.

[Host of Sorcery Podcast]

I know some people can't think this far in advance because AI has made things time windows very short, but what are you most excited for in the next 12 months?

[Dylan Fox]

Yeah, so 12 months is. It's actually. I feel like it's easier to predict like 24, 36 months than it is 12. I think over the next year we'll see a lot of a lot more consumer applications, hardware and software where voice is a core dimension. So toys, games, consumer electronics. And I think that's going to be really cool because I think that's going to, you know, like, one, like free us from our phones. Two, create this more futuristic environment where technology like, blends in more, you know, because right now a lot of technology. Like it's kind of like when you see a fridge that has the custom panels, you know, in a kitchen and like blends in and you don't notice the fridge, it's like, where's the fridge? Versus like the fridge that's just like stainless steel and this like object there. And I think that voice is a way to create like technology like that that just kind of exists because you don't need to be able to access it, you can like talk to it and it can talk back to you. So I think consumer is going to be an area where there's a lot of innovation. Were for example working on on device models, so models that can run like on a phone or on a really low powered piece of hardware, like a remote control for TV or something. And that's going to expand a lot of these applications. So that's part. And then I would say there's something that we all in the voice industry have to reckon with which is this. There's this interesting difference where every text based agent that you talk to, customer support in an app everywhere, at least my experience has been I have not encountered a single one that's trying to convince me that it's a human. It's always like, hey, I'm the AI sdr, I'm the AI support agent, I can help you with this. Voice is different where today when you're building a voice agent that you're talking to over the phone or something, for the most part you're trying to trick. The goal is to try to trick the human into believing that it's also a human. There's exceptions to that. Like tolans that we talk about where it's very clearly it's like an AI avatar, it's like a character that you're talking to. And I think that as an industry that's something that kind of have to figure out. I think right now you're trying to trick the user to believe you're a human because that's the way to mimic intelligence. Everyone's used to these really shitty voice experiences that just don't work. So they bail right away. And we see this. If our customers, if they're building a voice agent and you disclose up front that you're an AI, people just hang up versus if you don't, people continue. And I do think that will change and I think we're going to want that to change because I don't know about you, but I want to know is this a human or a agent? I had this really Weird call.

[Host of Sorcery Podcast]

Would you act differently?

[Dylan Fox]

Yeah, I would. I had this really weird call, and this is now, like, starting to trip me out. But we recently, like, had to move some stuff, so we had a moving company and I got a call and it was. It was like. I was like, after. I was like, that was a weird call. I was like, is that. Was that. I think that was an AI, but it was like. It was like, with an accent. The AI had an accent. It was like, really trying to trick me that it was a human. And it was just. The result was like, this is weird that I felt. And so I think that with voice agents in particular, there's something we still have to figure out about the UX of AI that you're talking to balance the, hey, this is intelligent, but I'm not trying to pretend to be a human and trick you experience, because if you ever talk to a voice agent and then like, two minutes in, you realize you're talking to A.I. it's just like a weird experience. And I think that's something that. That we still have to figure out.

[Host of Sorcery Podcast]

I feel like at some point we should just, like, assume everything is AI Though.

[Dylan Fox]

Maybe imagine you call some, like, medical helpline, and then you're, like, asking for help. When our kids were born, I used to call the, like, nurse line for the hospital to ask questions. And like, imagine now I'm doing that. And then like, two minutes in, it's like, you know, I realize I'm talking to AI and be like, wait, what is going on? Versus, like, no, I want to call, like, and talk to, like, a human nurse. You know, I want to know, you know, I want the options. But I don't think we figured out how to balance that today. And I think today when the market and the industry talks about a good voice experience, they think like, oh, yeah, it should, like, match like a human. And I think you can have good pacing and a good conversation. It can be natural. But, like, with a robot, that should be. There's examples in science fiction where that's been achieved, and I think we could figure out a way to do the same.

[Host of Sorcery Podcast]

So I know we covered a lot of topics. Is there anything that we didn't cover that you want to touch upon?

[Dylan Fox]

I feel like we covered a lot of it. Yeah. If you. Yeah, the new models that we're building that have the ability to take in context are, like, really cool. They're the first type of models that can Voice models that can do that. But no, we. It was. It was great to. To cover the full range.

[Host of Sorcery Podcast]

It seems kind of obvious, though. Like, if you had models at McDonald's when they don't know they're at McDonald's, like, that's crazy.

[Dylan Fox]

Yeah. Yeah. The voice models haven't been able to do that yet. And. And, you know, we. We shipped, like, really, like, the first one that can a couple of weeks ago.

[Host of Sorcery Podcast]

That's super cool.

[Dylan Fox]

Yeah.

[Host of Sorcery Podcast]

Okay, so as we close out, there's one question I have to ask. This is a bricks question because they're all about performance. Spending smarter, moving faster. I like to think that for personal performance, it's kind of who you surround yourself with. Some people say it's like the five closest people, like, who's either mentored you, who's a close friend, who's been inspiring. Who are those people for you?

[Dylan Fox]

Yeah, I really put it in, like, three buckets, like, family, friends, and then it's gonna sound so corny. But, like, really, our investors and, you know, on family, like, my wife's an entrepreneur too. She's a founder. And that's been, like, a blessing because, you know, that. That. Yeah. Always able to, like, get advice from her and talk to her friends. You know, I've been able to. To meet over the last couple years, like, other founders that are in the same stage and phase, and that's been amazing. And I think having, like, finding peers that you can just, like, be super open with and transparent with is super helpful. But then in terms of investors, like, we have an amazing group of investors that has really been along for the ride, you know, And I think about, like, Keith Block and SmithPoint that invested in our company and our series circumstances. They're, you know, operators from Salesforce. They've started their own VC fund. And, like, they're just, you know, so I think about, like, Steve from Excel, Steve Laughlin, Rebecca from Insight. Like, they're just always, like, pushing the company and pushing me and in. In great ways, and they're. They're amazing people. So that I'm not even trying to, like, be cheesy when I say, oh, no.

[Host of Sorcery Podcast]

I kind of feel like that's a hot take. I feel like a lot of founders are like, you know, I don't need the investors. Like, I'm good.

[Dylan Fox]

Yeah, they're. They're great. And I. I think we've been really, really lucky to have a great group of people around the company. So, yeah, I think friends, family, you know, investors have been. Have been. Have been super helpful for me along the. Along the journey.

[Host of Sorcery Podcast]

Amazing. It's a good place to end it. Well, Dylan, thank you so much. I appreciate you sharing everything on assembly in the early days to even the craziness that's happening now. And I hope pretty soon you can start translating dogs.

[Dylan Fox]

Yeah, we're working on it April 1st. Come, come check it out people.

[Host of Sorcery Podcast]

Yeah.

[Sponsor/Advertisement Voice]

Yeah.

[Host of Sorcery Podcast]

Ye. Thank you.

[Dylan Fox]

Yeah, cool. Thanks for having me on.

[Sponsor/Advertisement Voice]

Huge thank you to the entire Raise team for an incredible event. And thank you to Brex, MongoDB and Assembly AI for making this trip and series possible. If you enjoyed this conversation, you're going to love the rest of the Raise series with Tony Kim from BlackRock, Scott Wu from Cognition, Andrew Feldman from Cerebras, Rodrigo Yang from Samanova, Michael Hurlston from Lumentum, CJ Desai from MongoDB and many, many more. Like our hot takes that we did at a secret location that you can find on X, YouTube and Instagram. Subscribe to Sorcery on YouTube for more conversations with the people shaping AI and join the free newsletter. You can also do paid at Sorcery VC for weekly insights on AI, robotics, enterprise software, consumer semiconductors. Did I say AI? AI again. And everything that's coming next like funding announcements and all big things in tech. Thank you. Bye.