Loading summary
Transcript58 lines
- [00:00]
A
Hey, it's Oliver from this Week in AI, the brand new podcast from the team at Twist. We're dropping a sneak peek right here
- [00:06]
B
in your feed to show you what we've been building.
- [00:08]
A
If you enjoy it, join the community at ThisWeekInAI AI. Or find us on Spotify, Apple Podcasts, or YouTube. Like, I was talking to a friend of mine, she's an accountant, and she told me accounting is never gonna be replaced by automation. I'm like, what are you talking about? It's the first time we're really automating cognition as opposed to just automating the physical part of a job.
- [00:27]
B
70% of them think they'll have a decrease in job opportunities. Only 30% of Americans are worried in the same poll about themselves. So they all think it's happening to somebody else.
- [00:36]
C
I do think that humans will want to play status games. I think we'll find other jobs. I think we'll probably be doing less numerical and logical jobs.
- [00:43]
D
It feels like something very big is coming. The world doesn't appreciate that it's happening because most people are not very good at asking questions. You can taste the singularity. At this point, I can't even imagine the answer. End of this year is going to be shocking.
- [00:58]
B
Thanks to our friends at PayPal, the exclusive sponsor for this Week in AI. Try the payment and growth platform that's trusted by millions of customers worldwide. PayPal Open start growing today@paypalopen.com. all right, everybody, welcome back to not this Week in Startups, Not All In. This is a new roundtable I'm doing. It's called this week in AI. It's in the name, folks. Every week, three amazing CEOs, just like on All in or the VC Roundtable we do over at Twist. Three amazing CEOs who are actually building the future. And me, an investor in the space and an entrepreneur. Talk about the week's issues and sometimes the bigger picture issues. You can find out more about the podcast this Week in AI AI or if you want to see the YouTube channel, this Week in AI. AI. YouTube. And this is our seventh episode. It's March 3rd, 31st, 2026. Three amazing guests with us today. Jeremy Frankel is here. He's the CEO and co founder of Fundamental. They're building large tabular models for enterprises. They emerged from stealth as a unicorn just 16 months after founding 255 million Series A LED by Oak, with participation from Valor Battery, Salesforce and more. Welcome to the program, Jeremy Frankl. So, Jeremy, explain what your company is doing. And how it's going.
- [02:21]
A
So far things are going great. So what we are doing is we built a foundation model for tabular data. So what does that mean? So when people think about the AI boom or the AI revolution, everyone is thinking about LLMs and for good reason, right? Like ChatGPT, like create a breakthrough where you can now you pre train one model on the entire Internet and you understand language and you can use it to power thousands of use cases. But what's less appreciated is that LLMs really mostly solves unstructured data issues such as text, audio, video, images, coding, but they really didn't impact structural data. And structural data means everything that comes in rows and columns. So think about spreadsheets, databases, CRMs, ERPs, it's all rows and columns. And that's the vast majority of useful data for enterprises. And that part of the enterprise AI has never had its ChatGPT moment. And so this is the modality we're going after and for a variety of reasons, it's the one modality that acts very differently than others. And what we're building is really the ChatGPT moment for tabular data.
- [03:36]
B
So if I can repeat it back to you to make sure I understand this vision, it's what I always like to do with founders, see if I can repeat it back. You have large language models those are built as we all know, like guess the next word and transformers. It's based on massive corpuses of text based data. Generally speaking, you're building an LLM to focus specifically on tables and tabular data structures that we all know as a, you know, might experience an Excel sheet or a database, a Notion database, a, a SQL database. Am I correct?
- [04:13]
A
So it's not an LLM, it's a large tableau model. So it has a very different architecture than LLMs. Basically, if you look at the LLMs, as you said, they're built on being next token predictors. It's an autoregressive model. The problem with that is that if you look at the way transformers, for example, are applied to LLMs, they have a positional encoding as part of it. So the order of the sentence matter. If you change the orders of your, of your sentence in Cloud or ChatGPT, you can get a different output, but with tables you actually don't want that. And the reason why is imagine if you have a table with a million patients and you try and predict which one of them have cancer and your first column has the weight of the patient and your second column has the heart rate. Of the patients. If you switch the order of those columns and you first list the heart rate and then the weight, you shouldn't expect or you wouldn't want to expect a different output. But with LLMs, if you change the order of the data, you get a different output. And that's fine when you're writing an email, it's not fine when you, when you look at the deterministic outputs. And so that's what we've focused on.
- [05:15]
B
Got it. So large tabular model and LTM is how the first. Are you the first people to do this or is this like a known alternative to an LLM?
- [05:26]
A
It's very nascent. There are a few smaller companies working on it as well, more in an academics environment and academic setting. But we are the first like large company doing that at an enterprise scale.
- [05:37]
B
What is the benefit here? Is it because you'll have a better fidelity, better results, more trustable results than the problem we have with hallucinations in large language models? Would that be the reason to do this?
- [05:51]
A
It's very different use cases. So if you look at everything in the economy, for example, every time you swipe your credit card, one of the credit card providers has to make a split second decision of whether the transaction is fraudulent or not. When you work with retail, for example, forecasting demand, all of those. Or like for example, if you order an Uber, Uber has to make a prediction on the ETF of your driver. Each one of those stars is tabular by nature and it's predictive. But when you look at the way those predictions are being made, they still rely on traditional machine learning algorithms that predate LLMs. And those algorithms still do better than most LLMs at making those predictions. And so what we've built is a model that can essentially unify all of those use cases into one model to allow you to make much more accurate predictions than what you would otherwise be able to make.
- [06:41]
B
Genius. So the company is named Fundamental and your flagship model, the LTM large tabular model, is called Nexus. Am I correct there?
- [06:52]
A
Correct. Correct.
- [06:53]
B
All right. And we just had Perplexity CEO Aravind, who's really crushing it. He's one of your angels, huh?
- [06:59]
A
Correct.
- [07:00]
B
Awesome. Victor Riparbelli. I got your name correct, I hope. Riparbelli, welcome back to the program. You're with Synesthesia.
- [07:08]
C
Synthesia.
- [07:09]
B
Synthesia. S Y N T H E S I a. You're an AI video platform for business. 90% of Fortune 100 companies are your are customers. Already over 100 million in ARR. And you've raised over 500 million $4 billion valuation in. And we're seeing an incredible demo here. Explain to us how you're different, Nano Banana, the free services out there, ChatGPT's image generation, and why you exist as a company dedicated to just, you know, working on video models.
- [07:43]
C
So I think it goes we start the company 2017 way before any AI video tech actually worked. So we've been like quite a bunch of like different companies all the way up to 2026. What we decided on like five years ago was we're kind of looking at the early iterations of AI video, right? Which is the model that we're seeing right now, like Veo Sora, which just being discontinued. Our models, obviously very high fidelity, fairly inexpensive to run, assuming you're just like creating single videos and really getting to the point where you actually can't tell the difference, which unlocks a whole bunch of new use cases. But what we figured out five years ago was that the first iteration of this technology was not ready for primetime. It could definitely not make a Hollywood film, you could definitely not make performance marketing ads. But there was a very real use case in taking all the world's PowerPoint users and enabling them to communicate in video, as opposed to slide decks or documents, which, in 2022, when we launched, the first product was what everybody wants, right? People want to watch and listen to their content. They don't want to read that much anymore. And we essentially provide a way for PowerPoint creators to very easily switch to making video instead. And that has worked really, really well. And today we both have own models. We build voice models, we build video models, we have interactive models, so avatars you can actually talk to in real time, that's launching very soon. And then we also use a mix of the big models from some of the bigger providers to solve some workflows for our customers.
- [09:11]
B
And for folks who want to hear from you, three years ago, we had you on this week in Startups as part of our next unicorn series. Before you were a Unicorn, episode 1776. What a great episode number to have. Why did Chat GPT OpenAI shut down Sora? And why is Elon doubling down on video? He's been very vocal just this week talking about how video is the most important thing. Take us through your take on that. Obviously you were in video years before ChatGPT was even launched.
- [09:45]
C
I mean, I think it's kind of interesting that even a company like OpenAI founded by some of the smartest people in The Valley. Right. Some of the most accomplished people still had to learn the lesson of focus. I feel like it's one of those almost unteachable lessons that they had to learn the hard way. I think it's very obvious to anyone looking at the way that Anthropic is ripping right now that CodeGen is probably the most valuable near term use case for all these technologies. And I think OpenAI probably had a little bit of flying too close to the sun moment where they decided to do absolutely everything all at once. Which often the PowerPoint, it sounds doable, but in reality, I think anyone who's run a business knows that doing too many things at once is rarely a really good idea. And frotbik focused on no voice models, no video models, just like Codegen B2B, no freemium. And that has clearly paid off really well for them. So I think my take on what's going on in OpenAI is that they're like, let's cut all the side quests and focus on the market. That's really, really going to matter. And I think that's going to be probably screwing more towards B2B and it's going to be heavy on CodeGen and powering just this campion explosion of products that we're seeing being built with the vibe coding right now.
- [10:57]
B
I guess Claude's got people shaken. They've done. Or specifically it's got OpenAI shaken. They've added so much revenue. They've become such a darling. Jeremy, I see you smiling about this. This has become notable in the industry. Yeah. Jeremy.
- [11:11]
A
No, correct. And I mean like it's funny, I was at the, I was at the founder retreat a few weeks ago and everyone was just talking about cloud code. No one was mentioning anything else, the cloud code.
- [11:20]
B
So what founder retreat was this?
- [11:23]
A
It was a lightspeed event.
- [11:24]
B
Ah, lightspeed. Okay. The venture capital firm had. And that's fascinating. Nick Harris is back in the this Week in Family. You were on this Week in Startups. I remember this discussion August of 2023 as well, episode 1787. And so we, we got it right having you guys on early and we were talking about, and you were predicting just how important, you know, these data center was, these data centers were and that photonics using light instead of electricity to connect AI chips would be critically important. And you explained to me back on that program, Nick, that energy and data centers were going to be a major, major issue. And here we are three years later. Energy is the bottleneck, isn't it, Nick?
- [12:10]
D
Yeah, it's exactly the bottleneck. As a company, we've been focused on driving the future of computing. The central challenge that we're solving at Lightmatter is around how do you create a new roadmap for Moore's Law, for Dennard's scaling? These are rules that drove computing progress for our entire lives. I think about being a kid in the 90s, and every 18 months, you get an incredible new chip, more performance, all these things. That's over now. And there's only two ways that computers get better at this point. One is big computer chips. You put more chips in a package. Computer chips are getting to the point where Nvidia sells nearly $100,000 chips. So those chips are getting really big, and the size of the chip is going to keep growing. And the other way is that at any given time, there's the biggest chip you can build. So networking them together is the other piece. So big chips network together. This is the future of computing. It's the new Moore's Law. And we power both of these with our product passage. And we also, since we spoke last, started building lasers, which I never thought we would get into. But when you look at the photonics revolution, what's interesting is that you've got this device that powers all of the communication for these AI supercomputers, and it relies on the laser. It's kind of like batteries for electric vehicles. It's a really fundamental, huge part of the Baum, and it drives all the progress in how these computers are going to connect. One of the cool things to tie into the software piece of this is with photonic technology we've shown in research and so on, that you can actually 3x time to train. If you guys are watching Anthropic and their incredible tear, I'm hearing about the new model Mythos that's coming out. We can actually 3x with 3x faster time to train. Imagine if you have PE to the RT where you've got R times 3 now. So the rate of takeoff is going to go up like crazy. The first companies that adopt this photonic technology for linking up GPUs in AI data centers, the foundation companies, they're going to have an enormous advantage.
- [14:13]
B
Got it. And for the audience, again, my typical technique of repeating back so we all understand what you're doing. Nickmatter. Ethernet cables. That's how we connect computers. Typically, obviously, people are consumers are using WI Fi. You would never use that because it's very limited bandwidth. But Ethernet, which is typically copper wrapped in plastic versus Photonics, which would be made of glass, I'm assuming here, and fiber optics. Yes. And the throughput is radically different. Maybe you could explain.
- [14:45]
D
Exactly.
- [14:45]
B
Give us a bit of a primer and then you can sportscast what we are seeing on the screen right now.
- [14:50]
D
The way that AI supercomputers are built, you think about NVL 72 from Nvidia, they take 72 GPUs and they link them all together in a very high bandwidth domain. So here we're talking about petabit per second bandwidths within a rack that's all linked together in copper. What's really interesting about copper is it can't go very far. The cables have to be quite short. And so what you're seeing is the racks are getting packed as tight as possible. Now people are building racks that are a megawatt. So you have a rack that's a megawatt. You have to reinforce the concrete below it because it's so heavy that it's actually a load on the infrastructure. And you're building these custom racks that are just for delivering the cooling to these systems. That's kind of where people are at today. And the reason they're there is you have to bring the density because the copper can't reach very far. So just to give you an example of the delta and what we do, we just announced a chip with Qualcomm, where with each glass fiber, we're packing 16 wavelengths of light and we're pushing 1.6 terabits over a single optical fiber. That bandwidth is crazy. It's like 1600 houses worth of Internet. A normal house has one gig Internet, so 1600 houses. So copper really doesn't have very much reach. And the reason it matters is when you're building these AI supercomputers, if you want to have great performance, you want to link as many GPUs as you can tightly together. If you have optics like what we do, you don't need to put it all in one rack. You can separate it by a kilometer. It travels at the speed of light. There's very little loss in the optical fiber. And you can build giant systems that act like a single brain rather than a bunch of mini brains. With 72 GPUs talking in parallel, you could have thousands of GPUs working together on a workload. It drives interactivity, drives time to train. Both inference and training get a huge benefit out of switching from copper to optics. And we're kind of the leader in performance in this space.
- [16:47]
B
And just so People can conceive of a pet a bit. You're talking about thousands of 4K movies coming from Netflix every second. So this and probably 100 million high res photos from your camera, your library per second. So if you had a hundred, if we as consumers had 100 million photos, somehow in our photo libraries, Victor, you could be sending just hundreds of people's photo libraries. The entire, you could send the entire corpus of Netflix movies in 10 seconds. Yes, Nick.
- [17:27]
D
Yeah, exactly. And there's a kind of a cool analogy here. We have the chip M1000 that we announced last year. That chip is 114 terabit per second. So that's 114,000 gigabit per second. That's 114,000 houses worth of bandwidth. And a more interesting comparison is that that is about the bandwidth of the cables that connect North America to Europe for the Internet, the undersea optical cables. So we're building chips that have just an obscene amount of bandwidth and it's all needed to drive AI scaling.
- [17:57]
B
Jeremy or Victor, in terms of your data usage, when you hear about light matter and the impact it could have, what goes through your mind, Victor, because obviously you're working in video and your data centers. I'm not sure what your standard platform is and where you host, but maybe thinking ahead to the future, how do you think about what Nick is building?
- [18:20]
C
I think it's super exciting. So there's kind of two big ideas we found in Synthesia. The first one was like, as AI increasingly can generate data, the marginal cost of creating video, audio, also the content has got dropped to zero, right? Both in dollars, but also very much in time required and skills required. I think we're in the middle of that right now. But this is still video as we know it today. It's a broadcast medium. You make one video, you put it on YouTube and everybody watches exactly the same version of it. The second part of our thesis was always around when we invent new technologies as humans, we always invent new media for formats that are native to those technologies. Like the idea of a podcast or TikTok. Video wouldn't exist without modern technology. And for us the big question is like, what does video look like if you were to reinvent it in 2026 with all the new primes we have around us, right? We have LLMs with essentially intelligence on tap. We have offline video models that can create extremely high quality content. We have real time video models that you can interact with, advertise, you can talk to, canvases that can be drawn in real time and a whole bunch of other like, cool technologies around it. And so what we're building for and what we're actually launching, it's in private beta right now, launching in a couple of months, is real time video. Which is the idea that if you take one of our use cases, if you're a salesperson and you're doing a bunch of training to understand like the competitive landscape, a new product that you're launching, instead of just like receiving a video that you sit down and then you watch it and then you hope you understand it, it's going to be an interactive experience. It's going to be maybe first you consume some content, you go into an agent, you role play with it, it pretends to be a customer. You have to answer questions, overcome objections, then you go to another thing where we actually, in real time, draw a diagram of a customer's potential tech stack, how you're going to work with this, how you're going to integrate it. That's a very different type of video, which is almost closer to maybe a game or a website or something like that. But one of the bottlenecks here is of course, that if we're actually going to do this with video and we're going to do this with avatar models, we're going to draw things real time, that's going to take up a lot more bandwidth. It's also going to have much higher inference costs. And so the more we can reduce these, the more accessible this becomes. So I think in the next couple of years we'll see this becoming a new type of interface that's going to emerge. But for it to really take off and just be every interaction we have with the computer could be done with technology like this. We need the cost of serving that content to drop, like very significantly. And I think that's that that's the core of what the problem Nick's working on. So I think it's very exciting, Nick,
- [20:48]
B
when we have this ready for Victor to experience it and like it's. We could probably do a deal right now that he could be one of your beta customers because it would be amazing for Disney. I mean, I'm thinking in a consumer mind frame to sort of help the audience follow along here. But, but imagine, you know, Disney releases Mandalorian and they had done a deal with Sora to try to get the IP to work. Now imagine with Light Matter, being able to enable Victor's company to be able to make a short film with Grogu and the Mandalorian and you're talking to them in real time and it's making that in real time. That's just
- [21:27]
C
absolutely put some economics on that. Right. I think, you know, if you were to do that today, let's say you were to like personalize like a one hour movie for a kid from Disney, that would cost you like a lot of money, right? If you say like an 8 second clip with like a state of that video model costs like one or two dollars today, you can add that up for like an hour of eight second clips, right? That's not going to be sustainable. Within like a $15 per month it'll be.
- [21:50]
B
Yeah, if it was $6aminute. If we just made it like 6 bucks a minute, 120 minute. You're talking about $700 for a custom movie that you can live in.
- [22:00]
C
Exactly. And we're not that many years ahead of like I remember when I was a kid, right, and I had to like call my dad and ask him if I could download like a 10 megabyte file because it was like adsl and you were like mirrored on how much you would download. The idea of doing a video call for an hour with someone across the world, like that's an absolutely ludicrous idea.
- [22:16]
B
Right.
- [22:16]
C
But probably in like X amount of years this is going to be like completely normal. We're going to be just generating content in real time in front of people and we're going to be able to offer that at like, you know, within the subscriptions that these services charge today.
- [22:29]
B
Nick, you were going to add Jason.
- [22:31]
D
Yeah, we're actually busy building chips for a bunch of companies. We typically work with hyperscalers to build their own chips. Think about like the Google, Amazon, Microsoft, Meta type companies who are building their own hardware to do both training and inference. And then we also work with semiconductor companies, both GPU companies as well as networking companies. So those are the people we build for. We're building a ton of chips right now. So I would say in the next year and two years you're going to start running on light matter hardware. These will be in the new data centers. Think about like the Texas stuff. Yeah.
- [23:06]
B
What's the one? Not Starbase. Stargate. Another great film. Speaking of film.
- [23:11]
D
Yes, excellent film.
- [23:13]
B
Yeah. So there's a picture of, I think that's Stargate and what you see in the middle is that. Plus I think is, I think I was talking to Jensen or the CEO of Core we about this. Somebody on my team will tell me I believe this is Core weaves Data center. You have the cooling there. That bottom line that looks like memory chips in a motherboard. These are starting, these data centers are starting to look like giant motherboards. But I think those are the cooling apparatus where the contained water system. What do they call closed loop water system? Loop, yeah, Closed loop water system. So that was Crusoe's data center. I remember the CEO was walking me through it on a previous episode. That's the closed loop water system and the data exchange. Pretty compelling stuff, Jeremy, when you look at all this. Oh, by the way, Nick, Amazon making their own chips and I don't know if they're a customer, if they were, you could say so. But you gave us the sort of like Amazon's are called Trainium for training AI models and Inferentia for running inference models. Somebody at Amazon needs to go to branding school. That's a little too on the nose. Training and Inferentia, I mean, what did they do? They asked ChatGPT to come up with names. But these are going to be dedicated chips and I think Broadcom generally builds people their chips. Is that typically what happens? So if we explain it to the audience, you have Nvidia, they work with tsmc, they're making their own chipsets, they're the leader of the pack. Every single other hyperscaler. Tensors from Amazon, Tensors from Google.
- [24:51]
D
From Google.
- [24:53]
B
Then you have these Inferentia, Trainium and Mtia from Meta. Mtia from Meta. So explain to the audience why people are doing two different supply chains, Nick, and then we'll get to you, Jeremy, on your thoughts on this next wave and how that'll affect your business.
- [25:12]
D
Yeah, if you look at the incredible spend, I mean, they're over 100 billion a year. They're like 180 billion a year, I think is what Google announced they spent. I think Amazon was over 200 billion for the year. When you're spending that kind of money developing your own custom silicon is a little bit of a rounding error. So I think that they're really looking at these costs. They're trying to figure out how they can optimize cost and they think they can build their own solutions. Now building a chip is one thing, but building all the software and the ecosystem around it is another. And that's where Nvidia has had decades of experience building out the moat there with Cuda and everything. But everyone's trying to build these chips and the reason is that it's a race on the infrastructure point. People are trying to get power, they're investing in these micro nuclear reactors to go power the data centers, 100 megawatts each. So you get 10 of those and you've got a gigawatt data center. They're working on that power delivery, they're working on building the chips. They do their own. These hyperscalers are becoming very heavy duty infrastructure players. From cement to energy, all the way to chips and obviously the software stuff on top. There's just so much money in this space that they're all kind of making the bet on doing it themselves. And it's all in service of powering technologies like Jeremy and Victor's. It's really about the apps that run on top of this and getting the cost to the right point because the AI models are incredible. But we've got to keep driving down the token cost and driving up the inference rate and then we'll be able to keep unlocking incredible things like custom movies. And you're going to need blazingly fast interconnect for that.
- [26:47]
B
Jeremy, I'm assuming you're building on Cuda, which is the proprietary layer for coding and sending jobs to Nvidia hardware. Correct?
- [26:57]
A
Correct.
- [26:58]
B
And if you were to consider other platforms, other hardware platforms that were non Nvidia is there, have you considered that? And is there a path for you or would you have to maintain Cuda plus some other open source software? I guess there's some abstraction layers for CUDA now to get on AMD processors. So as the CEO, how do you think about where to spend your energy? Is it just too much to even consider other platforms? Or are you like Amazon, Meta and Google saying hey, we need to have two swings at Batman?
- [27:34]
A
I very much think that we are in the process of exploring different chips as well. You mentioned Trainium, one of the chips we're in the process of exploring. And the idea here is that we don't want to just be dependent on one hardware or one type of chip. Of course it comes with Cuda, gives you a lot of advantages and it's not easy to switch away from Cuda, but it's definitely something that we're in the process of exploring. And what Nick is doing is really exciting because the funny thing about everyone knows, but with video, you know about the amount of data that is being moved from one place to another. But people also don't realize that you have the same problem with tables. If you think about a table with 10 million rows and hundred columns, which is not even that big of a table, it's like what you have a billion cells that's orders of magnitude more than the context. The largest LLMs can even take in. The largest LLMs can maybe take 100,000 rows. But when you're working, for example, with banks on fraud detection, you're working with billions of rows. So you just need. And they're milliseconds matters. You make a decision when you swipe your credit card. You don't want to be waiting for 10 minutes before you get an answer. You just want to get an answer right away. And so the amount of data out there in tables is just massive. We've been talking to a few companies where think about every IoT sensor, every time you get some data that comes in some form of structured form. And the amount of data that they are dealing with is petabytes of data. And so being able to move data much faster and having lower latency and as Nick said, also lower cost will really be essential.
- [29:19]
B
Yeah, I mean, if you were to think about, I have these air things. I don't know if you guys care about air quality in your homes, but it turns out in your office, in your home, CO2 and radon, all this stuff, very important for health, very important for, like, cognitive function, especially CO2. So I have these error things and it's taking recordings in six rooms in three different houses. For me, the amount of data that just one person consumes, or let alone your whoop or your Fitbit, like how many heartbeats is whoop and shout out to whoop? They just raised money at a $10 billion valuation. Like, what's the data processing there when they have to do my sleep and my recovery and my run and my heartbeat. I mean, my lord, it's huge amount of data.
- [30:02]
A
Yeah, exactly. It's unfathomable amounts of data. And join the community at ThisWeekInAI AI or find us on Spotify, Apple Podcasts or YouTube.