SO MANY THINGS need to go right just so you can watch a TikTok! | E2215

Summary

This Week in Startups - E2215

Episode Title: SO MANY THINGS need to go right just so you can watch a TikTok!
Date: November 26, 2025
Host: Jason Calacanis
Guests: Gu Rao (CEO & Co-founder, Newbird), Mike Velardo (Subject AI), Alexander Wang (archival interview, Scale AI), with co-hosts Alex and Lon Harris

---

Overview of the Episode

This episode dives into the invisible complexity behind everyday technology experiences—like streaming a TikTok—and how AI is being applied in infrastructure, IT operations, and education. Host Jason Calacanis and team talk to Newbird’s Gu Rao about Hawkeye, an AI-powered agentic SRE (“Site Reliability Engineering”) tool, and to Subject AI’s Mike Velardo about scaling personalized education with AI. The episode wraps with an archival interview with Scale AI’s Alexander Wang, reflecting on AI’s past, present, and future, especially in the context of data labeling and autonomous vehicles.

Main Themes:

• The huge and intricate engineering required in digital infrastructure.
• AI’s role in automating and optimizing IT operations.
• Practical applications of generative and agentic AI in education.
• Reflections on the evolution and realistic future of general AI.

---

Key Segments, Insights & Quotes

1. How Many Things Must Go Right for a Simple TikTok To Play?

Guest: Gu Rao, CEO & Co-Founder, Newbird
Timestamps: 00:00 – 07:39

• Complex Layers of Delivery

  • "The moment you get on a phone and you're looking at a TikTok video, I mean the number of layers that are involved to deliver that... it's a very complex environment and people want more of this faster." (Gu Rao, 00:00)
  • Storage, data quality control, content distribution networks (CDN), edge caching—all must work seamlessly.

• Software & Infrastructure Example

  • "There's software associated with [storage]... quality control... no data loss, no integrity compromise. Then the distribution aspect... you don't want somebody looking at the video to hit the backend server each time—content distribution networks involved." (Gu Rao, 00:39)

• Troubleshooting Complexity

  • Identifying the point of failure (CDN, storage, backend, network) is a daunting, multi-layer problem.

Notable Moment:
Alex voices awe at the fact that Internet infrastructure stays online, despite such vast complexity:

"Is it more surprising that the Internet as we know it works today or is it more surprising when it breaks?... I'm kind of shocked that everything stays online most of the time." (Alex, 07:39)

---

2. AI for IT Ops: Inside Newbird’s Hawkeye

Guest: Gu Rao
Timestamps: 08:42 – 19:59

• Agentic AI in IT Operations

  • "An agentic system leverages generative AI... applies it to a very specific task domain... we leverage large language models, extract the reasoning capabilities and interface them with the complex array of metrics, logs, alerts, traces." (Gu Rao, 08:46 & 09:48)
  • Hawkeye acts as an “AI IT Ops engineer,” handling both SRE (Site Reliability Engineering) and traditional IT operations.

• Engineer vs. Agent vs. Copilot

  • Hawkeye autonomously finds, diagnoses, and potentially remediates IT issues. It can run both reactively and proactively.
  • "A co-pilot is like you have to distinctly carve out a body of work... what we're really trying to do is streamline your... IT operations where they're getting far fewer alerts or like 80% of the alerts are being handled by Hawkeye and the remediation is done. It's automatically resolved." (Gu Rao, 14:01)

• Context Engineering to Control AI Cost and Quality

  • "Anybody can go in and download a whole bunch of logs, paste it into ChatGPT and say what's wrong. ... That's called context engineering. That's all we do. We specialize in context engineering because... we don't want the customer to incur very large inference costs. More importantly, we don't want LLMs to come up with garbage answers." (Gu Rao, 15:57)

Notable Quote:

"More context—while these large language models have very large context windows, garbage in, garbage out." (Gu Rao, 16:10)

---

3. Hawkeye in Practice: Plug-and-Play, Reasoning Across Domains

Timestamps: 17:44 – 22:50

• Plug-and-Play AI for IT

  • Hawkeye doesn’t need extensive retraining for each environment; foundational models have seen common IT scenarios.
  • The system’s strength lies in “context engineering”—feeding the right slices of log/metric/alert data to the LLMs.

• Results After One Year

  • "We're achieving close to 90% plus reduction in the meantime to incident response and resolution from the RCAs that we're creating." (Gu Rao, 19:51)

• Autonomous or Consultative Actions

  • Hawkeye can both suggest and, if allowed, autonomously implement remediations (via code, feature flags, or PRs).

• Agentic Systems as Colleagues

  • "You should treat [agentic systems] like it has a Persona because these things are built out of human knowledge. ... I've had evaluations... where an SRE manager came to me and said, your product is doing really good. Hawkeye came up with an answer, but my SRE disagreed... both were correct." (Gu Rao, 22:50)

---

4. Pace & Plateau of AI Model Improvements

Timestamps: 24:16 – 26:29

• Slower Model Progress is Expected

  • "At some point everything starts to plateau... when you have billions of parameters... at some point adding more may not even make sense... you may just start getting weird answers. ...The knowledge in the models is reaching some sort of convergence." (Gu Rao, 24:54)

• Next Steps: Building Context & Tools Around AI

  • The “core of the brain” is ready; next phase is context, “external EQ” for models.

---

5. Hawkeye’s Proactive (Not Just Reactive) Capabilities

Timestamps: 26:29 – 28:00

• Retrospective analysis (root cause after an outage), real-time triage, and trend forecasting for preemptive action.
• “A customer will say, I’m planning to roll out new software… is my environment ready to absorb these changes?”
  Hawkeye responds, sometimes bluntly:

  "Hawkeye will tell you, hell no, it's not. You got a lot of work to do." (Alex & Gu Rao, 27:45)

---

6. Subject AI: AI for Education

Guest: Mike Velardo
Timestamps: 28:38 – 47:37

• From ‘Netflix of Education’ to AI-Driven Learning

  • Originally created cinematic-quality, short-form educational videos; pivoted to AI-powered, personalized classroom tooling.

• Flip Classroom & Teacher Empowerment

  • Subject AI enables “flipped classroom”—students cover basics with videos/quizzes; teachers provide high-value one-on-one.
  • "I'm really here to amplify the heroes of society, which are teachers and coaches... [Subject AI] allows for them to have more one on one interactions." (Mike Velardo, 33:45)

• AI Video Games and Engagement

  • Gamified, interactive, short-form curriculum: “All of our clips around five minutes or less now... 96% of our students complete at least one course." (Mike Velardo, 37:39)

• Teacher Tools: AI-Driven Grading and Planning

  • The platform auto-grades, plans lessons, and tracks student progress, aiming to triple student coverage while keeping teacher admin below an hour/day.

• AI Model Choice & Academic Integrity

  • Uses Claude, has used GPT and Gemini; models can be swapped modularly.
  • AI grading is “recommended”—teachers have ultimate say; for “teacher of record” cases, there’s an accredited human backstop.
  • "We want to make sure that students are actually learning and prepare for the next step... We don't want to be a credit mill." (Mike Velardo, 44:11)

---

7. Flashback: Scale AI’s Alexander Wang (2019)

Timestamps: 47:43 – end

• Alexander’s Early Journey

  • Raised $100M by 22; parents were physicists at Los Alamos. “I have a fun little history. I grew up in Los Alamos, New Mexico... both my parents are physicists.” (Alexander Wang, 50:32)

• What Scale AI Actually Does

  • Core business: providing high-quality labeled data for ML/AI projects, especially self-driving cars.
  • "Machines don't know what to do unless they have data that actually tells them what they're supposed to be doing... we are this data refinery." (Alexander Wang, 54:35)

• The Future of Self-Driving and Regulation

  • Alexander predicted regulation wouldn’t be a major blocker; most obstacles are technical and data-driven.
  • "Humans are terrible drivers and if we're going to allow 16 year olds to drive tanks down the highways at 100 miles an hour, maybe letting the smart computers do it's better idea." (Alex, 60:51)

• AI’s Impact on Labor & Progress

  • Autonomous trucks will take the “boring parts” of the job; AI augments, not eliminates, higher-value work.
  • "The automated truck driving systems, actually what they would do is automate the, the long haul middles ... and allow the current truck drivers to focus on these like higher value trips." (Alexander Wang, 62:17)

• AI Danger, Oversight, and Path to AGI

  • Alexander is skeptical of sci-fi fears; argues for practical oversight and responsible engineering, not dystopian anxieties.
  • "You only think that's not far fetched because you watched a lot of these sci fi movies... In reality, there's a lot of oversight over these machine learning systems." (Alexander Wang, 65:44)

  On general AI:

  • "It's very overblown, the timelines that people are talking about general AI happening... not even clear that if you have infinite compute, you'll be able to produce general AI. I think that's very unclear." (Alexander Wang, 74:19)

---

Memorable Quotes & Moments

• “I’m surprised any part of [this] infrastructure works... like when my train comes in on time, I’m really happy with it.”
  — Gu Rao (07:49)

• “We specialize in context engineering because... we don't want LLMs to come up with garbage answers.”
  — Gu Rao (15:57)

• “Treat agentic systems like a colleague. Sometimes they will surprise you, even out-innovate your human SREs.”
  — Gu Rao (22:50)

• “We want teachers to spend LESS time on the product and more with students. Let us do the grading.”
  — Mike Velardo, Subject AI (39:59)

• “Machines don't know what to do unless they have data that actually tells them what they're supposed to be doing... we are this data refinery.”
  — Alexander Wang, Scale AI (54:35)

---

Segment Timestamps & Topics

| Timestamp | Segment | |----------------|-------------------------------------------------------------| | 00:00-07:39 | The unseen complexity behind serving a TikTok video | | 07:39-19:59 | How AI can act as an IT Ops engineer (Newbird Hawkeye) | | 19:59-26:29 | Hawkeye's performance, context engineering, and evolution | | 26:29-28:00 | Hawkeye’s proactive capabilities in IT environments | | 28:38-47:37 | Subject AI’s evolution, tools, and impact in education | | 47:43-end | Archival interview with Alexander Wang (Scale AI) |

---

Episode Takeaways

• The Backbone of Modern Life is Vastly Complex and Fragile:
  A TikTok video relies on dozens of technical layers—any could fail.

• Agentic and Applied AI is Now Essential, Not Experimental:
  Whether diagnosing IT ops or personalizing learning, AI is moving from theoretical extra to operational core.

• Context is King in AI Ops:
  Feeding the right context to language models is the keystone to reliable, affordable automation.

• Education is Evolving Rapidly with AI:
  Making lesson planning, grading, and differentiation scalable is possible, but still needs a human touch and oversight.

• Long-Term AI Hype Should Be Tempered:
  Major advances plateau, more compute ≠ AGI. Oversight matters—and there’s more continuity than disruption in the AI story.

---

For Listeners Who Haven’t Tuned In

This episode is rich with insights for anyone curious about the "machine behind the curtain" in tech, how AI is being practically deployed in mission-critical business and education settings, what makes for robust and responsible AI adoption, and how even the AI titans themselves urge a blend of optimism, caution, and practicality.

Listen to this episode for:

• Real-life examples of AI augmenting teams (not just replacing them)
• Candid founder perspectives on deploying next-gen tools in enterprise and B2B SaaS
• A look back at how the AI conversation has changed in just five years

---

Transcript

[Gu Rao]

You know, what engineers do is troubleshoot, fix problems, they create problems and then they fix the problems they created. There's a lot of interest now in how to use AI, especially gen AI, and how that can ease the burden of IT operations. The moment you get on a phone and you're looking at a TikTok video, I mean the number of layers that are involved to deliver that, notwithstanding the physical infrastructure to deliver the audio and video content, but the infrastructure, software and physical infrastructure that it takes to store the media, it's a very complex environment and people want more of this faster.

[Alex]

Just for fun, can you walk me through some of the software that goes into making a TikTok appear on my phone?

[Gu Rao]

For example, the videos have to be stored somewhere so there's some quality control, so there's software associated with that. And then when the video gets edited and uploaded and stored, you want to make sure that there's no data loss, no integrity compromise on the video. And then the distribution aspect of it, then there's edge caching. You don't want somebody looking at the video to hit the backend server each time. And so there's what's called content distribution networks involved.

[Alex]

This week in startups is brought to you by Uber. Bad data equals bad AI. Your AI is only as good as the data it learns from. Uber. That's right. Uber AI Solutions now works with enterprises around the world to source, label, evaluate and scale real world high quality data from for every industry everywhere so that you can focus on building the next big thing. High quality data equals Smarter and faster AI. Uber.com twist perspective AI surveys they never capture what customers are really thinking. That's why we use perspective AI. Real insights straight from your customers. And the first two months are on us. Just go to getperspective AI/TWIST and CLA innovation takes balance. Our CPAs, consultants and wealth advisors can help you get from startup to where you want to end up. Get started now at claconnect.com/tech. Hey everybody, welcome back to Twist. And if you are in the United States, tomorrow is Thanksgiving. So I presume you are listening to us on the plane from the car or wherever you are. We're glad you're spending the day before the great holiday with us now on the show today, Newbird, a Twist 500 company that is building what it calls an agentic AI SRE. What does that mean? Well, it means they're applying modern AI technology to helping companies ensure that their cloud setup actually works. As the cloud becomes more complicated, there's More wires and tubes to connect and this company wants to make sure that you never get them crossed. Then Jason talks to Subject AI. It is a really cool company in the edtech space. Like some others, it's helping students have more personalized educational tools and also helping teachers get rid of the busy work so that way they can spend more time one on one with the kids. Then to close off, we're doing a little bit of a flashback all the way back to November of 2019 when Jason sat down with Alexander Wang, then the CEO of Scale AI, talking about the AI market and more. It is a fantastic time capsule and look into the prior era of of AI. Let's have some fun. I am paying a lot of attention to applied AI. Sure it's great to keep up with the newest models, but if you really want to understand the health of the AI market, I think you need to go and look where the rubber actually meets the road. So that's why we've talked to startups like Harvey, which is bringing AI into the legal realm. And today we're talking to another startup working in applying AI, albeit in a very different realm. New Bird AI. That's Neu Bird asked the question, what if we took modern AI tools and applied them to IT IT ops? And if you want to know why that matters, well, just think back to the recent AWS outage. Well, maybe these things are very important to the running of the economy. Anyways, let's learn some more. Please join me in welcoming to the show. It's Gu Rao, the CEO and co founder of Newbird. Hey, how you doing?

[Gu Rao]

Doing well Alex, thanks for having me.

[Alex]

Oh my absolute pleasure. I have to say that when it comes to hands on experience in IT Ops, I have precisely exactly zero. And I think a lot of folks out there, founders and all probably are in the same boat. So maybe the best place to start Gu is just waiting. Why do we need an AI IT operations engineer?

[Gu Rao]

Allow me to give you just a little bit of background. I'm an engineer. You know what engineers do is troubleshoot, fix problems, they create problems and then they fix the problems they created. And creating and fixing the problems for some way, shape or form typically happens at 2 in the morning and you don't want to wake up at 2 in the morning waking up to an IT outage saying that the website's not working or the payment processing system is down and you can't take transactions. So this is it. Computers are something that have moved us into the new frontier over the past 20 years and it's going to be more and more essential and critical to all of our lives. And so getting ahead of this problem and having infrastructure in place that can keep the operations running smoothly is of the utmost importance. So there's a lot of interest now in how to use AI, especially gen AI, and how that can ease the burden of IT operations.

[Alex]

Before we get into how you guys are doing that though, is it correct that the state of IT operations is one of increasing, ramping complexity? Because it feels like with the advent of like multi cloud setups and so forth, there's just a lot more pipes to keep connected. And so my impression as a layman in this context is that this is probably a problem that's getting harder and harder as time goes along.

[Gu Rao]

That's exactly what's happening as things mature and there's more demand for new and faster software, faster products, better features, complexity increases and people are moving at such a high velocity now, engineers in creating product, that there are many layers of technology, many components that interact with each other to deliver the features that you like. The moment you get on a phone and you're looking at a TikTok video, I mean the number of layers that are involved to deliver that, notwithstanding the physical infrastructure to deliver the audio and video content, but the infrastructure, software and physical infrastructure that it takes to store the media, it's a very complex environment and people want more of this faster.

[Alex]

Just for fun, can you walk me through, through some of the software that goes into making a TikTok appear on my phone, for example? So I think that's an interesting thought experiment to kind of talk through.

[Gu Rao]

First of all, there is the content creation and distribution aspect of things. The videos have to be stored somewhere. A lot of times this content is being curated by people that are brand specific. They have a tight control on their brand. They're not just putting any video up there. So there's some quality control. So there are software associated with that. And then when the video gets edited and uploaded and stored, you want to make sure that there's no data loss, no integrity compromise on the video. And then the distribution aspect of it. This now goes to. People are watching this on their Verizon or T Mobile and audio and video format doesn't need to get delivered in there. There's edge caching. You don't want somebody looking at the video to hit the backend server each time. And so there's what's called content distribution networks involved. You guys are all familiar with the cloudflare outage that happened a Couple of months ago. And so these things, there's many layers. Invol a video blogger goes out there and says, I put this content out there, but my customers can't reach it. Is the problem with the cloud distribution network is the problem with you running out of space on your AWS account? Like how do you even go troubleshooting these things? It's a complex problem. Takes many engineers to get involved and troubleshoot these things. Whether you're an engineer at TikTok or you're supporting the blogger's website.

[Alex]

Is it more surprising that the Internet as we know it works today or is it more surprising when it breaks? Because I feel like given what you just described described, I'm kind of shocked that everything stays online most of the time.

[Gu Rao]

Goo it takes a lot of hard work to keep things running. It's great that the infrastructure works. Look, I'm surprised that any part of infrastructure that works, like when my train comes in on time, I'm really happy with it. Kudos to the people that are managing that. And so it's taken many years for us to perfect how to get trains running on time. And if you were even look at the aircraft industry, right, or the air industry, that's a huge logistics issue. And so people have perfected that. And now we're getting into moving away from manually keeping things online to using AI to keep these kind of systems working.

[Alex]

But the complexity that we described generates so much information logs essentially, that I presume that AI is a good tool to be a kind of a first set of eyes on all the information that's coming in. Because me, the individual human, even if I replicate myself and set myself at 12 different desks, probably can't look at all the logs that are coming in from even a fraction of TikTok's overall digital footprint. To stick with that one example 100%.

[Gu Rao]

That's where Agentic systems come in. Let me define what an agentic system is.

[Lon Harris]

Sure.

[Gu Rao]

An agentic system is something that leverages gen AI to do what I think you called it applied AI, which is a really good term. So an agentic system takes generative AI and applies it to a very specific task domain, problem domain. And in this case, we're talking about IT ops. And so to your point, in IT ops, what an engineer has to go through when they're debugging an issue is they're looking at a complex array of data sources, they're looking at logs, they're looking at metrics, they're looking at traces, they're looking at alerts and each one of these have their own complexity, data structure, complexities, and there's a lot of this data. Data has always been the Achilles heel for an enterprise. So yes, while people can do this, it is tedious. Takes time and effort for people to go through logs, alerts, traces. AI can do this very quickly. All we do here at Newbird AI is exactly that. We leverage large language models, extract the reasoning capabilities and interface them with the complex array of metrics, logs, alerts, traces so that they can solve an IT issue much faster than humans can.

[Alex]

And this product is called Hawkeye. You guys call it an AI IT Ops engineer. I think you've also described it as an SRE in some capacity. Can you just explain to me the difference between IT Ops and site reliability engineers? Why you I think in one release said one, and one release the other. This might be a small point, but I'm just intellectually curious.

[Jason]

One of the themes we talk about over and over again on this week in startups is making sure you do your chores. I'm no expert on these things. I have some experience. Stephen Estes from CLA is an expert. Let's talk about being cash efficient. Tell us about efficiency and what you see in the in the top tier startups in your practice.

[Alex]

We're seeing kind of an interesting trend.

[Gu Rao]

Out there where companies aren't needing to.

[Mike Velardo]

Raise quite as much as they had in the past.

[Alex]

You really have to be careful as.

[Mike Velardo]

A founder to only take on as.

[Alex]

Much money as you really need. You got to do the forecast and.

[Mike Velardo]

You'Ve got to do the modeling and you got it dialed in and get it right.

[Lon Harris]

Otherwise you're going to end up either.

[Mike Velardo]

Not raising enough capital to get to where you're going and you're going to have to go get venture debt or go back, have an extender to the.

[Alex]

Round, or you're going to give up.

[Gu Rao]

Too much of the company because you.

[Mike Velardo]

Just didn't recognize how much money you actually needed.

[Jason]

Yeah, very important to get this stuff right, folks. And that's really a bummer when startups don't do things in a button up. I always have a great partner, a good partner to have on this adventure while things change. My friend Stephen over at CLA visit claconnect.comtech and don't forget to mention that your boy J. Cal sent you that. Claconnect.com Tech Start Today.

[Gu Rao]

Site reliability engineering was a coined term by Google for very large enterprises where there are specific engineers focused on maintaining site operations. Now Depending on the size of the company you talk to, there are dedicated SREs that are completely focused on making sure that the IT operations are running up smooth. These are guys that respond to things like pager duty alerts. Like goes down, they get an alert, they have to go in and resolve the issue. Now as you get into more mid market, there's a blend. A lot of times engineers that are working on product or in DevOps share the role of maintaining their site operations. So they're doing double duty, so to speak. These products, agentic systems, address both use cases. So to talk about agentic systems in the context of site reliability engineering appeals to very large enterprises. But at the same time we want to make sure that the same technology is helping mid market as well.

[Alex]

So essentially I should think about SREs as very, very specialized IT ops engineers, normally the largest enterprises. But if you're going to sell it to the mid market and up, you're going to want to speak to both the IT ops and SRE community a hundred percent.

[Gu Rao]

A lot of times you know, the engineering team that's writing code, they're on call duty. Like if the site goes down, this engineer, you have your project to do, you're working on your coding, but you're also on call support if something happens. And so you want this product to be able to address the needs for both.

[Alex]

You guys describe Hawkeye as an engineer versus an agent. Is an engineer in this context. Kind of like three agents in one training trench coat.

[Gu Rao]

The terms and how these things resonate with people, it's still evolving. Everybody has a different expectation on what the term is called, what it does. Is it a co pilot, is it an agent, is it something autonomously? Am I interacting with it? These are all things for people to figure out how they're going to live with. Agentic systems, we call it an engineer because Hawkeye, initially when it's deployed, it is autonomously and asynchronously figuring out what's going wrong with your IT ops and then figuring out what the issues are and creating remediation solutions. If somebody wants to interact with IT and say, look, there's nothing going wrong now, but what if I were to do this? Can you go figure this out? People can do that too. Do you call that an engineer? Do you call it an agent?

[Alex]

Engineer is a specific term for a general category that I would say is distinct from agents, I think copilots. If you go back to the early days of the post ChatGPT era, when Microsoft was talking about copilots, There are little assistants. They help you. I think agents to me feel more like single shot tools that go out and do a thing. Like, I'm going to send my agent to Amazon to buy myself candles. The reason why I kind of like that you guys are not calling this an agent is that it does feel to be like a discrete set of systems that I can work with but also work for me even when I'm not directly tasking them with A, please go do A on site B. Right. So it does feel different to me in a positive way.

[Gu Rao]

You hit the nail on the head, Alex. And look, this is a little fluid, but that's the feedback we're getting too. You're right. A copilot is like you have to distinctly carve out a body of work for it to go in and do or you're asking it to cross check the work that you've done, which is kind of not what's happening here. What we're really trying to do is streamline yours as an enterprise's IT operations where they're getting far fewer alerts or like 80% of the alerts are being handled by Hawkeye and the remediation is done. It's automatically resolved. I'll give you a simplest use case if somebody's wondering like a very large company here in the Bay Area that operates in the cloud, they run in aws. Obviously cost operations is a big issue for them. All they want Hawkeye to do is see if their today's spend is projected to be greater than 5% of yesterday's baseline and if so, why?

[Alex]

You mean greater than 5% differential?

[Gu Rao]

Differential from yesterday's baseline. That's it. Simplest of use cases. You would think like that sounds easy. But then you realize what engineers have to do. They have to log in, look at every instance that's launched, which instance changed size or maybe it's even instances. Maybe somebody's consuming more storage capacity or maybe somebody's doing a lot of egress traffic and downloading a lot of file like it. It is not that easy. And most importantly, it's kind of boring at the end of the day when you find the issue. So you just kind of let these agents do this on their own.

[Alex]

This seems to me expensive because when I think about using AI direct to do a lot of stuff, I just began to think about inference costs. And you guys talk about having integrations with Elasticsearch, Prometheus, IBM, PagerDuty, MongoDB, Reddit, Snowflake, everybody. Which means that if I'm going to send out Hawkeye and I'm a big cloud user who has a lot of different things to keep an eye on. It feels like that Hawkeye would need to ingest so much information and process it that it would be relatively heavy or expensive to run goo. Am I over indexing on gross margin fears here or is it actually as.

[Gu Rao]

Expensive as I think are dialing in on one of the core problems that we need to solve for Anybody can go in and download a whole bunch of logs, paste it into ChatGPT and say what's wrong over here? What you've actually done is you've isolated the problem down to a certain area and you're asking for a solution. I'll answer your question. The term for that is called context engineering. Quite honestly, more context. While these large language models have very large context windows in garbage out. To answer your question on cost but doctor were to charge you by minute, okay, just assume and you go in unprepared and you say I don't know what my problem is. It could be in my head, it could be in my back, it could be in my legs. Well, you're paying a lot of money. So what you will do then is before you go to the doctor you're going to say I'm going to ask myself what is my biggest issue? Isolate it, go to the doctor and say I have this problem in my knee right here when I touch it like this. And you'll get a very good answer from the doctor that's called context engineering. That's all we do. At Newbird. We specialize in context engineering because of two reasons. We don't want the customer to incur very large inference costs. More importantly, we don't want the LLMs to come up with garbage answers or what's called hallucinations. When you give a marketing document to a chatgpt and say I did this campaign last week, next week I'm going to reinvent create a new campaign. It'll do a great job. But you can't do that with IT systems because you want Hawkeye to give you exactly what's going wrong with your website and why it's failing. Not a what if not a possibility. Yeah, exact.

[Alex]

Does the system have to be trained on a specific company's setup or is it kind of something that you can plug into any cloud or IT environment and it will automatically know how to look around and observe the latter?

[Gu Rao]

It's plug and play. And the reason for this is we believe these foundational models have seen so Many different IT scenarios that the specific ailment you're facing is not something that hasn't ever happened before. We're not solving for that minutia of possibilities. So then all we have to do is take all this knowledge that exists in the LLMs and apply it to your context, your data. That's what we do here at Newbird. We are context engineers. We pick the right alerts, logs, metrics, traces so that the LLMs can diagnose the problem using reasoning. And we apply a different kind of reasoning methodologies. We have these models. Models argue with each other, cross check each other, consult knowledge basis to see if the reason that it's coming up with is in the realm of something that's happened before. It's a complex system to build agentic systems, but it all relies on good context engineering.

[Alex]

I've always felt that once we could get AI models to kind of fight amongst themselves as discrete entities, we would have some really positive friction that would yield better end results. And I think that's one reason why I'm so bullish on building up more compute capacity. Because I think it would be great to to be able to be wasteful with inference because then I think we can do quite a lot. But that's down the road. You guys launched Hawkeye General availability like a year ago.

[Gu Rao]

Yep, a year ago. Yeah.

[Alex]

So sitting here 12 months later, goo, how's it going?

[Gu Rao]

Pretty good. Look, we intentionally launched this with limited availability so we could learn with key design partners. We picked enterprises of all shapes and sizes, some very large logistic companies, some very large financial institutions, pharmaceutical companies, manufacturing two all the way down to low end of the mid market. Why? We want to see if the product resonates well with the whole spectrum. What type of problems do you have? At the very large enterprise, a lot of noisy alerts. At the very low end they want even more surgical because they're more tied into DevOps. What part of my code change is actually causing this issue? And so we've learned a lot, we've had a lot of good success. What do I mean by that? We're achieving close to 90% plus reduction in the meantime to incident response and resolution from the RCAs that we're creating.

[Alex]

Meantime of incident to response. So that means that if it used to take me 10 hours to get from problem to solution and now it takes me one.

[Jason]

You can't make your product better without listening to your customers. But how are you supposed to actually figure out what your customers want? You can send out a Survey, of course, but people just want to give the right answer. In that case, they're not really giving you their honest opinion. Well, at this week in startups we actually found the perfect answer. It's called Perspective AI. Their expertly trained AI conducts one on one interviews with your users based on your prompts and questions. Just tell their system whatever you want feedback on in simple, simple language and they take care of the rest. We've learned so much about our audience since teaming up with prospective AI. Reginald for example, is a founder who listens and wants more quick hit, faster paced content. And an anonymous guy from Canada gave us tons of helpful feedback about some audio syncing issues we've been having and we fixed them. This is the kind of feedback that's invaluable when you're a founder who is product obsessed like you should be. Sign up today and get started in just a few minutes at getperspective AI Twist and you'll get your first two months free. That's getperspective AI slash twist.

[Gu Rao]

Exactly. And by the way, that's not uncommon to and it's not just time, it it takes an army of people. Look, you can't imagine that any one engineer is an expert in networking and kubernetes and storage. That's not possible. People have expertise in different areas and thankfully these LLMs sort of have cross domain expertise. They have a lot of knowledge but they need to be told what to do. And that's where the context engineering comes from. In the hardest of problems we've been told that Hawkeye is able to save people a lot of time and at minimum tell them all the things that it did look at, which are something that the engineer, the on call engineer, shouldn't look at and dial into where the problem could be. Even if Hawkeye doesn't have access to all the information, sometimes it doesn't. It'll say I've taken care of all of this, this area, I don't have visibility. Focus on this.

[Alex]

Does Hawkeye recommend methods of resolving the issue or can it actually go in there and make the changes or updates corrections that I need to resolve it autonomously?

[Gu Rao]

Both. So it depends on the customer's comfort level. Typically this has been the case. When people deploy Hawkeye they're looking at it from a consultation perspective. Tell me what the remediation is. Give me a root cause. It will create the remediation report, code change or corrective actions. But people may not give it right access to take the action and it can submit a PR and somebody would have to hit approve. That's normally the case in some areas where feature flags and things like that are involved, because that's low hanging fruit. People will just go take the feature flag, enable or disable it.

[Alex]

But it sounds like maybe a good way to track how smart our overall AI intelligence is becoming is as more and more Newburgh customers allow Hawkeye to make those changes, it'll imply that there's increasing base model intelligence and also increasingly sophisticated applied AI techniques that you guys decide to make this entire thing feel more like magic and less like integrated systems.

[Gu Rao]

100%, Alex. And the other thing that I'll say here for your audience, and I'm not talking specifically about Newberg, I would imagine that this is on in all facets because I think agentic systems are going to be a thing in the enterprise, not just in it, but in all facets. Marketing, sales. And people should treat agentic systems as not, you know, off the shelf software where you have a known outcome. You should treat it like it has a Persona because these things are built out of human knowledge. So what do I mean by this? You have a chance to interact with it. So it's not a binary answer. You're not evaluating an agentic system for saying it's good or bad. There's a range in between. So a lot of times these agentic systems can come back with an answer. You have time to interact with it, give it feedback and say, go do this. I like what you did, but you fell short here. Go make these changes. It is like another employee. You asked me, how have things been going? I've had evaluations come in where an SRE manager came to me and said, said your product is doing really good. Hawkeye came up with an answer, but my SRE disagreed with it. Okay. Turns out both were correct. Hawkeye found an issue which was correct. SRE found an issue and the SRE had never thought of that. So it offers you different insights into your environment. It is bringing diversity, it's bringing a different chain of thought because it's been trained from so many different scenarios. And, you know, the manager was pleasantly really happy with that. And ultimately solving both issues improve the site ops multiple fold.

[Alex]

I think it's funny how we go from deterministic computing systems to more probabilistic. We do have to do more talking to it. Like, great, try again. I spend a lot of time with my ChatGPT instance being like, excellent, now please do it without being lazy this time. Thank you. Very, very fundamentally, are we still seeing the pace of AI model improvement that you were hoping to see because it does feel a little bit like from where I sit, that we are seeing improvements in Chinese open source models, we're seeing some improvements in American and European closed source models. But it doesn't feel as fast goo as it used to. And I'm curious if I'm just wrong about that or if you guys are also kind of noticing a bit of a.

[Gu Rao]

No, no. It's at some point everything starts to plateau and so I think you're acknowledging the fact that the rate of new features is starting to reach limits. And I think this is expected in any new system. You're evolving so fast and you're putting out features so quickly and, and you're training these models and when you have billions of parameters that these models have, it takes time to train them but then the number of parameters are limited. And at some point also having more parameters may not even make sense. I mean you may just start getting weird answers. So in that sense the knowledge in the models is reaching at this point some sort of convergence or. And it'll keep increasing, but the rate of increase, I think it's expected to not be at that same angle of trajectory that it was.

[Alex]

Well, that's too bad. I was really hoping it was going to keep going straight up a while there because I'm, I could use some more free intelligence to kind of layer onto myself. You know, I would love to be.

[Gu Rao]

Smarter but see the intelligence is going to come from external systems is what I'm saying. I think there's enough knowledge in these models to do a lot of good work. Now it's on external tools and external context. We have these things called MCP Model context Protocol.

[Alex]

It's an anthropic project.

[Gu Rao]

Yeah, the core of the brain. If it's steady, then augmenting its intelligence or sort of call it if it has enough iq, let's get it EQ you which is the external knowledge that it would need to do awesome tasks. And that's where I think the next two or three years in AI, at least as it comes to, as you put it, applied AI is going to be around building the context and the support system around this really great set of brains that we have.

[Alex]

One last question goo before I let you go, we talked about there being a problem. Hawkeye notices it, figures out what's going wrong and maybe either suggests or executes a remediation. Are you guys going to eventually go, go in front of the issues and begin to do like Proactive scanning and checking to see what might break in someone's IT infrastructure. Because I feel like that would be a natural next step. But I may again not be correct.

[Gu Rao]

We actually do that. When somebody interfaces with Hawkeye, there's three phases to their journey. The first phase actually is retrospective. The opposite of what you said. Hey, I want to try Hawkeye. Last week I had an issue. I did what every person on the planet does was I rebooted my cluster because I don't know what the issue is. I want to get out of it. I got out of the issue by rebooting what was the actual issue. So retrospective analysis. And then Hawkeye will say, I'm glad you got out of the issue, but this is what you should have done. Number two, real time analysis. Your how is my environment running? I have an alarm. What's the issue? And that's the primary use case. Big use case for Hawkeye is what you said, which is trend. What could happen if things are progressing the way it's progressing today, what will happen next week? A very good use case for that. A customer will say, I'm planning to roll out new software. Here are my changes. Is my environment ready to absorb these changes?

[Alex]

Hawkeye will tell you, hell no, it's not. You got a lot of work to do.

[Gu Rao]

Yeah, it could be that. Look, you have this much reserved memory capacity, but your pod spec is requesting more capacity than you have allocated. Your pods will fail to start. Sounds like reasonable answer to me.

[Alex]

Sounds like a very reasonable answer to me. It's Newbird AI if you want to go check it out. Neubird AI and do you promise to tell me when you're closing your next major round of capital before we left? So I'll leave that to you.

[Gu Rao]

We are very well funded, fortunate to have great investors. We're funded by Mayfield, Microsoft. We have enough money in the bank that we don't need to. But then, you know, we're always talking to investors, and there are strategic relationships that we're forging without disclosing anything. I think there'll be some good news down the line soon.

[Alex]

All right, well, we'll see you on January 15th, give or take. That's what that sounds like to me. Goo an absolute treat. We'll talk to you soon.

[Gu Rao]

Thank you so much, Alex.

[Alex]

All right, next up on the show, we have Mike Velardo from Subject AI Jason, this is an AI education company. Started off by making courses for online consumption, but has since pivoted into more tools for the classroom. We've talked to other companies in the space, like Magic School, but this one is blowing up. So let's talk to Mike. Mike, welcome to the show.

[Mike Velardo]

Hey, thank you so much for having me. Longtime listener, first time callers, excited to be here today.

[Jason]

Great note for startup founders. Michael, for some really intelligent reason, decided to put me on his updates to investors. This is a really great technique for founders. We train people to do this at the launch accelerator, which is you make two lists, your investors who you want to keep updated. So that's your investor update and then your dream investor investors. These can be the same email or slightly different with your existing investors. You might say, Alex, hey, here's where we're going and here's some challenges. This is the problem. Here's the legal issue we're dealing with. Okay, right. We're getting sued by a former employee. We got a patent dispute. Okay, great. That's for the internal, like 20 investors, right? Then there's the prospective investors. Now this is just where you shine, right? Hey, this is the third quarter in a row that we've had 20% on average growth. We are now on pace to hit 5 million in AR, 10 million in AR by this year. And we have cash in the bank. And we've just reached Infinite Runway because we're profitable. And then somebody like me gets this email. Now I'm BCC'd on it, and I'm like, wait a second, I've invested in 600 companies.

[Gu Rao]

Did I invest in Michael's company?

[Alex]

Oh, no.

[Jason]

Then I do a search and there's like an email from Michael pitching me that I never got back to from four years ago or a dm. But anyway, well done, Michael. You've been doing this technique for some time. Or is this the first time you did it with AI? Like everything else, the old saying is so true. Garbage in, garbage out. Without the right training data, you're not going to get great results. But my pals at Uber, they're now working with enterprises all over the world to source, label, evaluate, and scale high quality, real data for your startup. Uber is one of my favorite companies on the planet because they demonstrated their ability to scale and to build great products first in the rideshare market. Then they figured out Uber Eats, and then they moved on to autonomous driving. And all of this was only possible because of a deep understanding of how to collect, label, understand and analyze data. Now with Uber AI Solutions, you can put the team and tools that turned Uber into one of the world's best companies to work for. You Partnering with top talent and experts from Uber's global network. What better partner to help you scale your business than the company that organizes over 36 million rides every single day? Book a demo right now by going to uber.com twist that's u b e r dot com.

[Mike Velardo]

I'm doing it a little bit more often, but I do have to give you a shout out. I believe we were mentioned on a previous Twist episode when we rebranded from a meal learning to subject and got subject.com and you mentioned it as startup of the week and seven figure domain. Now we didn't pay that much, but I do believe on the market it could be worth that. So I always appreciated the way you gave us the nod then. And that's when I actually reached out via Twitter or X and we're really excited that finally we're here together.

[Alex]

How much did you pay? How much? You can't, you can't say.

[Mike Velardo]

Not undisclosed amount, but you know, low six figures. So we're really excited about that.

[Jason]

Three or four hundred grand for that is a pretty good deal. I have begin right now.

[Mike Velardo]

Less than that I can confirm. So less than that. And we also have Subject AI now as well. So we really take a lot of pride in our branding and name recognition and you know, we want to be synonymous with elite premium education for any age in any part of the world.

[Jason]

Hey, so why don't you show us what you're building. You can share your screen if you like and just walk us through the website or the product. I don't know if we gave you a heads up to do do that, but might be nice to just show the product a little bit and what's working because I know you started pre AI and then you wound up here with building AI tools.

[Gu Rao]

Right?

[Jason]

So yeah, tell us what you're working on.

[Mike Velardo]

I could quickly voice over and really give you up to speed. So we were really the Netflix of education subject. I started this during my MBA experience. It was 2019. I was at UCLA. It brought me to Los Angeles. 2020, I got thrown into Zoom University. The worst experience of my life. 70 grand a year to sit on a Zoom lecture like this. Everyone, you know, had their screen blacked out, slacking each other the answers. And you know, in Los Angeles, it was pretty extreme. Lockdown. Couldn't even eat outside at a restaurant for some period. So what did you do after six, seven hours of Zoom? Well, you watched Tiger King, Last Dance, all these Netflix shows. That was the saving grace and got us through that time period. So my Co founder. And I really said, hey, how can we make education feel like Netflix? How can we help students be engaged? I'm a millennial. Imagine a Gen Z learner or now Gen Alpha in the middle of North Dakota and Nebraska. They might not have nearly the sports that UCLA was providing me. And so that was our original passion. Premium cinematic quality. I'm in our office in Los angeles. We have four production studios here, 10,000 square feet where we do all our filming. But then since the launch of AI, we've really graduated past that and really providing much more adept personalized tools for students and teachers. Save them thousands of hours a year and allow all of them to feel like they're learning from their favorite teacher every single day in the classroom.

[Jason]

So in the same way Waymo and Uber and Neuro are trying to make like the perfect driver, you're trying to make the perfect teacher with AI.

[Mike Velardo]

Yeah, 100%. I'm really here to amplify the heroes of society, which are teachers and coaches. You know, I had a lot of great teachers and coaches growing up. And our goal is to give teachers the time to spend with their students in person, making magical moments happen. And, you know, we're seeing that firsthand now with subject AI, they're able to service three times the amount of students on a daily basis by using our product product. And that allows for them to have more one on one interactions like this. While subject AI can be the core experience in the classroom.

[Jason]

This used to be called adaptive learning in the industry. It was like really hard to code, et cetera. But it's gotten easier. Yeah. To do adaptive learning.

[Mike Velardo]

Yeah, adaptive learning, or commonly referred to also as flipped classroom, where you walk into a classroom and every student is engaging with subject videography, personalized AI tooling, and then the teacher is doing small group or one on one instruction with students who are behind or way ahead, making it much more impactful. And actually on the Andreessen Horowitz podcast, Mark and Ben have a really great episode on education and they talk about this is very commonly known in the industry. The highest impact on a student's journey is how much one on one tutoring time they get. The problem is most people can't afford one on one tutoring. But when you work within the public school system and you allow subject to be that core methodology in the classroom, now every student has a chance of getting one on one time with every one of their teachers.

[Jason]

So flip classroom is the technical term. You learn the repetitive stuff on your laptop and then when you're stuck or you're excelling. Then the teacher comes and says, oh yeah, hey, you're way far ahead. Here's some additional coursework for you to go on. Or if you're behind, let's get, let's break you through whatever the blocker is. Yeah.

[Mike Velardo]

Oh, 100%. I mean think about it from a first principal standpoint. Why should a teacher be doing the same lecture three times a day, year over year? And how much variance is there? You know, depending on your previous day, anything going on in your personal life, the teacher may be performing at their best or worse on that given day day. How can we give the teacher the time to focus on that one on one instruction where they love it? They get so much energy from talking to students in person. And the mundane as well as the grading and feedback is all taken care of by AI. And there's no excuse in this world of AI now to not be able to provide these teachers much more supports so they can have a much better work experience as well.

[Jason]

You just show us because I got. Yeah, I want to see it.

[Mike Velardo]

You can see it's very much like a Spotify Netflix esque vibe. When you go through the LMS here, we really want it to always feel like premium consumer technology or social media idea. Our goal is to bring the best trends from those industries into education. And you know we did the shout out earlier about you being an Uber investor, me being an Uber ex Uber founder. You know that's what Travis and Emil who are huge, huge inspirations for me. Emil is actually one of our investors. They brought the best people to work on transportation across all industries. Our goal is to do that in education and we're super excited about that. You could also turn into light mode was a lot of the teachers like when they put on a projection screen. But we'll go into dark mode here year so we're super excited about the product right now. So you could quickly see here how we're able to engage students like never before. This is all premium cinematography filmed in our LA studios. You know most of the legacy players in our space are either no videography or leveraging 30 plus minute videos. Everything we do is short form. You know especially Gen Z and Gen Alpha. They're looking for 30 seconds to a minute clip. All of our clips around five minutes or less now. And you can see a quick 10 second clip here.

[Alex]

Hello again, I'm Ms. Holly and I'm.

[Jason]

Back to discuss two important ideas to.

[Alex]

Help us understand stories better.

[Mike Velardo]

The central idea really engaging teachers and Videography. But videos are only a small part of the course. As you can see on our left hand panel. This is what a typical course will look like. We'll have one video and then a variety of quizzes, check for understanding and exams. That way you could be able to take the entire course asynchronous and use it in a live classroom setting to be able to get through the curriculum them. One of the most exciting pieces though, with AI now is now we have all of our courses have video games, which really engages the students. Right now, 96% of our students complete at least one course on subject. That's record high engagement in the industry. And so that means when a Student starts@ Subject.com or Subject AI, 96% of the time, they complete at least one course. And these new AI video games are a huge driver of this.

[Jason]

All right, so you're doing a. A video game of some type to teach. Teach storytelling and fiction.

[Alex]

Yes.

[Jason]

Got it. Wow. And these are all made with AI, I take it, or you just hire.

[Mike Velardo]

Everything with AI it's wild. Oh, I got it wrong. So that's not a good look for me. And I'll quickly show you one last teacher tool so that way you can get a good snippet of that before we jump back in. You know, the educator portal and everything we do for educators is absolutely critical for our success. We live to serve teachers. We live to serve teachers in suitable students, but teachers are the core methodology to be able to get through to these students. When you work with a district, it's very uncommon that a student would, you know, revolt to a school and say, hey, if we don't get subject, I'm dropping out. Typically, not a lot of agency around the student voice. The key is to have teachers say, hey, my 20 students are crushing on subject. You need to get subject memberships for your students as well. And so we're really big on empowering teachers. But this is all done with AI and this is why we're saving so much time for teachers. Today. We do all the grading, lesson planning and scoring and feedback for the them in our console here. And so quickly you could see which students aren't passing their course within seconds, you're able to see, oh, wow, I have five pages of students not passing. How can I do something in class today to help?

[Jason]

So it's, it's really got the whole classroom information data across all of their Project Pro Progress in the Adaptive Learning. And you're just using a chat interface for the teacher to solve these problems.

[Mike Velardo]

Pretty quick, 100%. And this is what's really powerful as well because you know, we are in a highly regulated industry. Every state has its own nuance. You could be able to go into our product and get state specific suggestions, scoring and feedback, lesson planning and worksheets within seconds. And so we're really excited about the advent of AI there and how we've been able to really accelerate our growth. So started with the premium videos, made a more short form every single year and now everything we're really doing from here on out has been AI native. We really made the transition in early 25. We launched subject AI in June of this year. Big inflection point for Subject.

[Alex]

It's really, really, really interesting. I'm kind of blown away how in depth it is. It feels like you've replaced the entire school framework with a series of videos, quizzes and so forth. That's highly automated. Does this mean that individual teachers can therefore better serve more students at once? Because I feel like if they're just focusing on one to one, the actual kind of quote, class size becomes less, less meaningful.

[Mike Velardo]

Well, so what we're really excited about, and we see this in the data, one, we want our students to spend more and more time on the product because that shows they love it. And right now, now our students are spending around three hours a day on the product. They love subject, but teachers, we want them to spend less time on the product. We want them to be with the students in person. We want them to be hanging out, making those magical in person experience moments happen. And so while we've seen teacher usage stay under an hour over the last two years, we've seen triple the amount of students served. That's exactly what we're going for. Higher leverage, more opportunity for teachers to have better access to student in person experience. Experience. And let us do all the mundane admin work, let us do the grading, let us do the scoring and feedback. Let you do what you love.

[Alex]

That's actually the only thing you've said thus far that I, that I've kind of like freaked out about a little bit is the grading element of this because. Well, actually maybe a better question to ask is what are the underlying models powering the teacher side of Subject AI?

[Mike Velardo]

We've used a variety of models. Right now we're leveraging Claude, which has been fantastic for us, but you know, we've used ChatGPT, Gemini. We're constantly looking to be able to see what's the best and we have easily on our code base being the Ability to swap in and swap out. I think the big thing for us and you know, especially when you look at legacy competitors in our space, you know, and courseware and such, a lot of them have had the ding of being click for credit. You know, a lot of multiple choice true false answers. We never wanted that adage with subject. And so previously we were having, you know, a teacher having to grade so many of these open form essay responses. Now with AI, this is speeding things up and allows for us to have a lot of authentic assignments while keeping up mass speed and delivery to these students to get instantaneous feedback.

[Alex]

Is there anything really lost though when. When Claude grades my assignment versus the teacher who knows me and perhaps even the. I can imagine a teacher's syllabus being partially verbal and not entirely written down and therefore ingestible into the AI model itself. And so like therefore the grading system might not entirely match up with what the teacher wants. It just seems like a place where I personally, I wouldn't put as much AI versus on the teaching side.

[Mike Velardo]

Yeah, so when we do digital curriculum, everything we do is a recommended grade. And so the teacher has the final say no matter what. When we do teacher of rec record, we typically are the final grade, but there's still always opportunity for them to amend edit the grade if they have some feedback for us. And so we're always working with our districts. At the end of the day, the customer is always right. And so whatever the district wants to be able to do for their students, we're here to serve them.

[Alex]

Can you actually explain teacher of record to folks? Because I don't think that's actually a term of art that most of our.

[Mike Velardo]

Listeners know totally and especially any industry jargon, acronyms, etc. It's always like, okay, let's explain it to the greater masses. You know, when I first started in this industry, I had no idea what these terms were meant. So basically we're accredited by two major governing bodies. Wasc, Western association of Schools and Colleges. This is very common. You know, Stanford, usc, Cal, they're all WASC accredited. There's six regional bodies across the US and they're also Cognia accredited, which is an international accreditation that took, you know, many years to get. And we're really proud of that. With those accreditations, we have the ability to service as teacher record for our districts. And that means that without a credentialed teacher in the classroom classroom we could step in on the back end and provide a credentialed experience through subject.com or subject AI and so with AI, that's made it a lot more scalable and high leverage. And so when they do teacher record, it's typically not a credentialed teacher. In the district. We're providing the teacher of record credentialed instructor on the back end.

[Alex]

And this allows for AP classes and such that might have a low student count, but then they can be brought to schools that wouldn't otherwise be able to support them, was my understanding of it.

[Mike Velardo]

Yeah. I mean even we were the first cryptocurrency accredited class in the country. That's exciting. Taught by Chad Copeland, Brandon CO Copeland. Financial literacy is a big one as well. It's a high usage for us. Brandon Copeland teaches the class. He teaches that same class at UPenn. So you have an Ivy League instructor teaching the class for high school students across the US we're super excited. But then you also see and this is what we're really passionate about. And you know, I came from a small town, Cary, Illinois, a lot of small town schools. It's hard to fill all, especially STEM courses, science and math. And so when you lose a science teacher in a rural community, it's very hard to backfill that. And so we can step in and help support that district. District by providing a four credit course with our teacher of record. AI.

[Alex]

What's the impact on student ability to learn and be college ready? There's been a lot of reporting lately about how people get into pick your brand name school with the unable to like divide fractions and very basic things that I thought we'd handled. How much can you guys help get our graduating high school students up to snuff?

[Mike Velardo]

I feel very confident in our ability. I think that's the big transition we're going through with education technology in general is how can we be able to be more part of the solution and making sure all the rigor is behind it and not allowing students to easily cheat their way through platforms. And so that's why we take academic integrity to the most seriousness and make sure that students aren't just passing. We don't want to be a credit mill and that, you know, that's again one of the negative terminologies in our industry. We want to make sure that students are actually learning and prepare for the next step. You know, there's over 40 million adults in the US who have college debt but no degree. We never want to be a part of that opportunity. That is super aggravating to me. It motivates me every day to make sure we're helping students either go to top college or go direct into the workforce. And that's what we're doing now with the career technical education product that we're debuting in 2026. So hey, you're hearing it here first.

[Alex]

So you're going to help people be essentially STEM ready out the gate. Wow, that's fantastic.

[Jason]

How do you make the decision to go sell into schools, which is a slow, arduous process, versus just doing what masterclass did or just go direct, like just go direct to consumers. How do you make that decision?

[Mike Velardo]

So in 2021 we were all direct to consumer and we were only AP classes and we got a lot of great momentum, raised a great round from Kleiner Perkins. Annie Case, our partner there, love working with her. Then over the we saw a huge dip in signups because naturally direct to consumer and K12 education, a lot of folks aren't active in the summer. And so we were like, hey, do we want to always engage with this? Especially we're part of K12. B2B is going to be the stickier business model. But also which is really important from the mission standpoint, it's not just affecting affluent families. Whereas B2C AP was almost all affluent families. When you're selling in the districts, you know, we work with some of the most poverty stricken areas of the country, which means the world to me. You know, like we're in some incredible districts.

[Jason]

How long do they take to like.

[Mike Velardo]

Actually get on board when we sell the product?

[Jason]

Is it a year sales cycle? Two years? Because there are, you know, in the.

[Mike Velardo]

Beginning and so we really struggle when we pivoted and you know, it was a couple tough years, but those years made us stronger. And now we're seeing sales cycles of under six months, which is really great. But in the beginning, learning to sell to schools, especially being outside the industry, there was so much to learn and we readily acknowledge that and we definitely had to really learn the hard way. But now the last couple years we have an incredible acceleration momentum. It just took us, you know, all of really 22 and 23 to understand the sales cycle cycle and understand who even the buyer was. What were the key stakeholders, what was icp. And now we have great alignment on it and we're full scale mode.

[Jason]

Awesome. Listen, continued success. Thanks for coming on the program. Everybody go check out subject.com or subject AI either one. Any positions you're hiring for.

[Mike Velardo]

So really talented folks who are willing to be in office five days a week in either Los Angeles or Austin. We have two offices now and we're super excited about that. And the biggest thing for us, I would say, as top level engineers and data folks, we really want to continue to improve the efficacy of our data and we believe that's the best for unlocking more sales and more product delivery. So data and engineers, please find our way. We have a lot of job openings ready to rock. We'd love to work with you.

[Jason]

You're an in person guy.

[Mike Velardo]

All about in person, tell people why. We launched in 2020 is a pretty sensitive HR environment, to say the least, with COVID going on and whatnot. And so we were very flexible for the first few years and, you know, we struggled. We didn't, we didn't have a lot of momentum, we weren't scaling quick. And so I really said, hey, hey, why don't we try just running the company our way rather than trying to please anyone. And I come from a sports background and so, you know, hey, when you're playing high level sports, everyone's training together in person. So we went full hardcore five days a week in office and the numbers skyrocketed and it's 100% not the right fit for everyone. I'm totally empathetic of that. But for subject, if you want to work with us, we're really passionate about building in person as a team. And, you know, one of our core values is championship, sports, team mentality. And so we love being able to build together.

[Gu Rao]

Love it.

[Jason]

Awesome. Continued success and we'll see you next time.

[Alex]

One of my favorite parts of being here at Twist is that we get to look back at figures that are currently making news. Back when we had them in the studio, before they were famous, before the tens of billions of dollars. One of those people is Alexander Wang. Now, he was in charge of scale AI and then eventually sold about half that company to Meta and became the new God of its superintelligence team. But back in 2019, on episode, episode 1005, Jason had Alexander in the studio to talk about all things AI. And keep in mind, this is several years before ChatGPT came out. Please welcome back to the show, It's Lon Harris. Lon, we're gonna do another flashback and this interview is a straight banger.

[Lon Harris]

Yeah, this one was great. I think what's so interesting about this is 2020. It was like, I think spring 2020 was when all of these apps like ChatGPT and all the, like stable diffusion, like the first generation of image to text ones were blowing up. To me, that feels like when all on, like the ground level became aware of the AI revolution is real and it's happening. And these apps are like, they exist now and you can use them. So it's fascinating to go back to a time before that and hear people talk about what we were all going to be talking about within a few years, but really like none of us understood at all. Like Alexander at some point said, this is sort of explaining to Jason, like, what is training and how do you train a machine learning system and like, like how do they learn? And all of this information is now embedded in all of our minds because we've been thinking about it every day for years. But back then it was like very theoretical. You could see Jason still like wrapping his mind around it in some ways.

[Alex]

Oh, absolutely. I mean, asking the difference between AI and ML and where does the data come from and what do you need to do with the data? Can you share data between providers? Alex was like, absolutely not. But let's jump in. Lon, where do you want to start?

[Lon Harris]

Right away they talk about, you know, Alexander being only 22 years old, old already raising 100 million. I like that Jason felt a little bit of kinship with him as a person who had also been a young buck founder who got asked a lot about his age and being so precocious. So let's take a look at that.

[Jason]

I guess the thing that most people would think is remarkable is candidly that You've raised over $100 million in your last round of funding. That's a lot of money.

[Alexander Wang]

Quite a bit.

[Jason]

It's quite a bit of money from Founders Fund that I think are 22 years old.

[Alexander Wang]

22, yeah, 22.

[Jason]

That's annoying to be young and successful because then every interview starts with your.

[Alexander Wang]

Age a little bit.

[Jason]

It's annoying. I had it happen to me if IT were like 23 year old publisher of Cyber Server, 25 year old publisher of Silicon, and I was like, why does my age matter now? I tell you, when you hit about 35, 40, they don't mention it anymore because they're like, wow, you're 40, you should be doing interesting things or be successful in the world. But you've been running this company since you were how old?

[Alexander Wang]

19.

[Jason]

19. How did you get into the game?

[Alexander Wang]

I have a fun little history. I grew up in Los Alamos, New Mexico. So both my parents are physicists and they worked at the National Lab in Los Alamos.

[Jason]

Yeah, tell people about that lab.

[Alexander Wang]

It was the, it was the lab where the atomic bomb was originally built. So the, the Manhattan Project Started in Los Alamos. It was very secretive at that time. And what do they call that lab? Los Alamos National Lab.

[Jason]

Yes, Los Alamos National Lab.

[Alexander Wang]

It's pretty boring.

[Jason]

It's pretty boring.

[Alexander Wang]

Yeah, yeah, yeah.

[Jason]

It's a.

[Lon Harris]

God.

[Jason]

Government sponsored lab.

[Alexander Wang]

Exactly, yeah. Totally government funded. And then growing up in high school, I did a bunch of programming. I did all these coding competitions. I was getting recruiter inbounds in high school. So after high school I actually came out here to work. I worked at this company, Quora, for a couple of years.

[Jason]

Yeah, we know it. They do the Q and A site.

[Alexander Wang]

Yeah, Q and A site.

[Jason]

How'd you get that job? You just applied and they saw your code and they were like, okay.

[Alexander Wang]

They recruiter inbounds because I was an anonymous person on these coding competitions. And then.

[Jason]

So you could just go into a coding competition? Nobody knows your age?

[Gu Rao]

Age.

[Jason]

Do you teach yourself how to code?

[Alexander Wang]

I guess the Internet, you just looked it up.

[Jason]

You found courses online on YouTube or.

[Alexander Wang]

It's hard to remember. I think I just googled around. Anyway, I worked at Quora for a couple of years doing engineering, infrastructure, etc.

[Jason]

No college?

[Alexander Wang]

No. Well, then I went to college after that.

[Jason]

Ah.

[Alexander Wang]

I went to MIT and then got basically bored after a year and started. Started scale.

[Jason]

So you left?

[Alexander Wang]

I left, yeah.

[Alex]

I remember lawn back when I was young enough to be considered a rising something or other. And now? Now I'm just a middle, middle, middle, middle person. But Alexander Rang is still quite young, so his career trajectory kept going straight up law. If I had given you $100 million when you were 22, how well would you have spent that money?

[Lon Harris]

Oh, not at all. I mean, this is always a fascinating thing for me, looking into the tech business where everybody gets started young and it's all these like, prodigal talents who started coding when they were 13. Like, I had no idea who I wanted to be or what I wanted to do when I was Alexander Wang's agent. This video, probably still not even Alexander Wang's age today, but definitely not. When he made this video, I was like, I just graduated college. I was floating around, maybe I want to go to grad school. Maybe I want to be a journalist. Maybe I want to write screenplays. I don't know. I was working in post production in Hollywood doing subtitles for HBO DVDs when I was AJ. So no, I. I don't think I would have been in a good position to raise a hundred million dollars and decide what to do with it. One more thing. I wanted to Pull up there from, from this opening though, I thought a really fun little tidbit was Alexander comes by being a prodigally gifted STEM guy pretty naturally. Both of his parents are physicists who work together at the Los Alamos Laboratory. That of course famous, if you've seen the film Oppenheimer is where the Manhattan Project was and they were doing the Trinity test. So you know, he grew up in that kind of like a hard science environment around people who were doing, doing like fascinating high level research.

[Alex]

Next up, Alexander explains how scale provides extremely accurate data to the autonomous vehicle companies. This interview Lon really spent a lot of time talking about self driving because, you know, in the pre GPT era we didn't think a lot about, you know, chat bots and that sort of thing. AI agents weren't even a phrase. We were trying to get cars to stay on the road. Take a listen.

[Alexander Wang]

The one that has really captivated the world's attention is autonomous vehicles. Right. It's a compelling example because first nobody likes driving, but also driving is very unsafe. There's a lot of risk in driving.

[Jason]

Sure.

[Alexander Wang]

And so the lot at stake. Yeah, exactly. And so the captivating sort of machine learning model is one that can take in all of the camera data and other sensor data from the vehicle, understand everything that's going on around it. Something that's very easy for you or I, but currently, or at least before machine learning was, was very difficult for machines and then can determine the best path to take and figure out how to drive on its own, basically.

[Jason]

Got it. So we see the lane markers, double yellow markers, double white markers on the highway. We know keep the car between those two lines exactly as smoothly as possible. We see somebody deviate from their lane into ours, we know to slow down, give them some room, maybe they're drunk. Machine doesn't know that inherently. We have to teach it that. Exactly what does scale.com do the that Tesla and Waymo don't already do? Because they're solving that problem. Do they use your software and do they need to.

[Alexander Wang]

The core problem as you just laid out, is that machines don't know what to do unless they have data that actually tells them what they're supposed to be doing. Right. And so what that means is one of the huge bottlenecks for machine learning is data ends up being like data that tells these algorithms, tell these models what they're supposed to be doing. That's where scale comes in. What we are is sort of of this, this data refinery, if you will. We accept a Bunch of raw data from our customers. We go through and process it and we sort of, we tell the machine what it should be doing. For example, given an image taken by a self driving car, we would outline, these are where the people are, these are where the cars are, these are the lane markings, et cetera. So that over time these algorithms can learn those things.

[Jason]

And I see you are highlighting cars, you're highlighting people. Exactly. And the machine is figuring out, okay, that's the approximate shape of a dot Dodge pickup truck, that's a Toyota Prius, and these look like the silhouettes of people. But that's a human telling the machine that's what it is for now.

[Alexander Wang]

Yeah, exactly. So the, the core way that our whole pipeline works is that a lot of work is done behind the scenes by machines and our own, our own AI models originally. And then humans basically give input and correct mistakes to make sure that, that the end data is extreme. Because that ultimately is what's important for the safety of these systems and for low bias, et cetera. All these things that are needed, imperative for machine learning to be.

[Jason]

So you would go to a customer, so they would give you videos of their cars driving and then you would annotate it for them and put that data into a database somehow.

[Alexander Wang]

That's exactly right. So for example, if they gave us a video like this, you'll see originally the first step was a human drawing a box.

[Gu Rao]

Yep.

[Alexander Wang]

And then a machine learning model that's already pre processed through all this data has determined the path of that vehicle over time.

[Jason]

Right.

[Alexander Wang]

And then we confirm that all this is correct and then send that data over to the customer and they train machine learning models on top of it.

[Jason]

Got it. And this is how, I guess one of the cars got fooled. Somebody drew on the ground like an arrow turning and a car followed the arrow, which a human would do too. But they basically drew a turning signal to see if like it would fool a self driving car. And of course it just did.

[Alexander Wang]

I didn't see this news. I would believe that that is how that's what would happen, basically. Yeah, yeah.

[Jason]

And that's what would happen to a human, by the way. So I thought that was the stupidest prank ever. They're like, look, we can fool a machine that's driving cars to make a wrong turn.

[Alexander Wang]

In a lot of ways. They will have, they will have some of the same challenges that humans have been driving.

[Jason]

I was wondering, when do you think we'll have a self driving car in a major city like San Francisco driving a Major route. What year would this be possible? 2030 and over or under 2030. When do you think we'd first see this?

[Alexander Wang]

This is the million dollar question, which is when?

[Jason]

Or it's actually a trillion dollar question.

[Alexander Wang]

Trillion dollar question.

[Jason]

Let's be real where we're at.

[Alexander Wang]

Fundamentally, the technology is getting better and better every year. The algorithms that perceive their environments are getting a lot better, like asymptotically better every year. And then the algorithms that figure out what the car's supposed to do, the planning algorithm, et cetera, are also getting better. So it really is, it's sort of only a matter of time before we get to the point where these kinds of routes are possible and we'll live in a safer world.

[Jason]

So that seems to me that you're thinking less than 10 years from now this will be happening with regulation all that counted in.

[Alexander Wang]

I don't think regulation is necessarily going to be that tricky.

[Gu Rao]

Why?

[Alexander Wang]

There is precedent for like if you think of when autopilots first came about, I think there's precedent for how to think about a lot of these things.

[Jason]

You mean autopilot in the airplane sense or in the Elon sense?

[Alexander Wang]

In the airplane sense.

[Jason]

So the FAA and whatever regulatory body is really like, okay, we get it. Autopilot works better than a human.

[Alexander Wang]

Exactly.

[Jason]

It's pretty obvious. Plenty of room up there to operate when you're up at 30,000ft. A lot less room to operate when you're going through the Tenderloin. There's six people in the middle of the street though.

[Alexander Wang]

The challenges are a bit different, but I think there's press and for how to think about these things. I think the technology, once it gets good enough, will be clearly extremely good. And so I don't think it'll be that big of a deal.

[Jason]

Andrew. So you think 10 years, maybe less.

[Alexander Wang]

I'm excited for the self driving future.

[Lon Harris]

What's so interesting here? I'm mean, obviously Jason having to be sort of told or instructed about, here's how we're training these AI models. You know, like the cars record the video and then they send it to us and then we annotate it with all the like helpful things that the computers need to learn and then send it back. It's interesting just that Jason needs to be sort of like this was probably the first time he encountered, you know, the differences between what Tesla's doing with their full self driving and what Wayo is doing with their lives. Radar and like just thinking all this stuff out again, it's become so second nature for us today. We've all been in Waymos, we've all, you know, tried this sort of out or at least everybody in the tech industry sort of been thinking about it. Whereas it was still so new and they were on this, you know, horizon of this really exciting new time which they then go to talk about. Like Jason asks him what Alexander says is the trillion dollar question, which is basically predict the future of self driving cars.

[Alex]

And turns out the future is many players several years down the road all doing incredibly well. Waymo expanded their geographic footprint recently. The Tesla Robotaxi project is doing well. Zoox is in Vegas. Everyone's working with everybody and it's going very quickly. So this AI thing I think worked out lon all this hype from 2019.

[Lon Harris]

There is one other very prescient moment where they're talking about over the next 10 years. Of course this was already six years ago. How often are we going to be in self driving cars in 10 years? Alexander didn't want to like set an exact, he didn't want to be nailed down. But one thing that he does say is that he predicts that regulation's not going to be that tricky, that like we have some precedent for the idea of self driving vehicles. He mentions autopilot in planes, but you could also think of like trains or other things that are on tracks or monorails or whatever. At the time that must have seemed like a very bold prediction. Like you think cars are going to drive themselves around city and cities aren't going to want to like obsessively monitor that for safety and they're not going to like regulate the hell out of that. And I think he's been sort of proved right. We're doing these pilot programs. People sort of roughly believe in this technology and trust it and they're, they're able to sort of roll out these tests to more and more cities all the time and expand it and there's some pushback. But I really don't feel like the regulation of self driving cars has been super onerous and it's stopping them from collecting the data that they need.

[Alex]

To the contrary, regulations for self driving cars in the US are lacks that many Chinese self driving companies are training here. So there you go.

[Lon Harris]

That was a pretty remarkable thing to predict in 2019 when I think most of us would have expected cities to crack down very hard on this sort of thing.

[Alex]

Yeah, it turns out that humans are terrible drivers and if we're going to allow 16 year olds to drive tanks down the highways at 100 miles an hour. Maybe letting the smart computers do it's better idea. Lon, the next thing I want to grab is speaking of presence and kind of getting things right. The two of them discussed the importance of helping humans do higher value work. But now when we talk today about AI agents increasing model intelligence, we're always thinking about what's this going to replace in the human labor pool? Is it blue collar work, white collar work? But often we're seeing AI today turn up in very high economic value areas. Coding, legal work, that sort of thing. And so when Alexander Wang discusses where he sees the AI market going, it's really interesting. Take a listen.

[Alexander Wang]

I think there's like helping people focus on higher and higher value. You work, I mean that's sort of like the core of human progress in some sense I strongly believe will be the actual story of AI and machine learning. And it'll have to happen more and more and more for us to be comfortable with it. A great example is like truck driving. So there's, there's all these automated truck driving companies.

[Jason]

Yeah, lots.

[Alexander Wang]

We work with a lot of them, Embark, Ike, et cetera. The naive view is that hey, they're just, they're going to automate truck drivers. And like if you look at the map of the states, like truck driving is a top profession in a lot of states. So it seems really bad. But actually if you look at the system as a whole, there's a shortage, there's a national shortage of truck drivers. Truck drivers in the United States.

[Jason]

And the median age is like 50 or something crazy.

[Alexander Wang]

There's this kind of like instability in the market because of all of this stuff. The automated truck driving systems. Actually what they would do is automate the, the long haul middles of these truck drivers.

[Jason]

Boring parts.

[Alexander Wang]

Which are the boring parts arduous that displace people from wherever their homes are, etc.

[Jason]

Yeah.

[Alexander Wang]

And allow the current truck driver drivers to focus on these like higher value trips that are sort of like warehouse to a meeting point or whatnot. Yeah.

[Jason]

Drayage to the factory or even the last mile. I mean who knows, like maybe these trucks will change their form factor and be half the size, be automated. And when the truck gets off the road, the same truck, Instead of using 18 wheelers, we might just use smaller mid sized trucks that'll be electric and solar powered so you have more of them when they get off they become the delivery truck and they just automatically start delivering. Delivering could be a much better model.

[Alexander Wang]

These sort of like the Introduction of machine learning to improve the efficiency of the economy. It'll be slow because of how in general free market economics work. It'll take effect in areas where there's an acute problem today. It'll happen in those places first. And it'll allow the current jobs that exist to become higher value, more impactful.

[Lon Harris]

Yeah, I feel like his truck driving example here is really good. It's a very smart illustration of the larger point that I think makes it easy to sort of visualize. Because what he's arguing is, you know, they're not going to fire all the truck drivers overnight. It's not like AI's here. AI could drive the trucks. We don't need you anymore. It's like there are different kinds of truck driving. There's the kind of truck driving where you're in a straight line down the highway for like three days in a row. And then there's the more intricate. You're going through a neighborhood, you're going around some sharp turns, you're, you're the last mile where the truck needs to go to actually make the delivery or what have you. And so he's saying, you know, like, well, I would take part of that. The like long haul, boring part that's sort of uneventful. You don't really need a human for it. And that's the part that makes a human drive three days away from their house and then be away from their family for the whole week to get that haul done. And so he's like, well, you could have the AI do that part. And then local truck drivers take care of the more intricate parts where it helps to have a human human and they get to stay in the same place. So it's like AI makes everything better and takes care of the stuff that we don't want human staff to do. And it's like, I don't, I don't know if that's actually how all of this stuff is going to play out. But that strikes me as like the best case scenario, like the utopian vision for how AI would integrate into our lives.

[Alex]

I'm a little more pessimistic about human labor than that, but I'll take the bullish scenario. Why not? I want to grab one more thing from this, from my end, Lon. And then if you have anything else, we can go to that. But they discussed the dangers of AI. And what I found very interesting in this was just how much unbothered Alexander is. I mean that really in a completely non joking way as a real positive. I Mean, I, I do think we went through a period of AI excitement, a period of AI concern. And now mostly we're talking about economic concerns. You know, is the data center build out a bubble? But we're not talking as much about like is AI being used and exist potential risk to humanity. And so to hear Jason, this is back, you know, five, six years ago, taking the tack that he did surprised me because this is not how he talks today. So take a listen to this.

[Alexander Wang]

Well, we believe the true narrative will be extremely positive actually, versus the current narrative, which is like AI and AGI, et cetera, are going to take over the world.

[Jason]

There is a possibility that AI could get out of control at a certain point with exponential computing. That's not far fetched, that it could do something crazy and stupid.

[Alexander Wang]

You only think that's not far fetched because you watched a lot of these sci fi movies.

[Jason]

If you were to train an AI to work on a drug to kill cancer and you didn't program it properly, it could create a drug that was too aggressive because you didn't tell it well in the process of killing cancer, please don't make the person blind or all these other things. So you could just forget some edge case and some general AI might think if you said to the general AI, you should work on things that make the human species better and goes, okay, yeah, let's collect cancer. And then it's like, oh yeah, let's cure this communicable disease. Great. The best way to cure communicable disease is to kill everybody who have the disease currency so it can't be communicated. This sounds far fetched, but there will be instances where they will make the wrong decision, right? Or it will be just too slow of a ramp up for us not to catch it.

[Alexander Wang]

The thought experience always go like, oh, you'll make an errant comment to an AI and all of a sudden it'll take over the world and do something that you really don't want it to do. I mean, in reality, like there's a lot of oversight over these machine learning systems.

[Jason]

Right?

[Alexander Wang]

Right now there's like tens, hundreds of people who look at these models. They look at all the data that comes in and out, they like analyze everything and they try to figure out, okay, what is this model doing well, was it doing poorly and how do we adjust to that, et cetera. So I think that could happen in a world where it's like we have low oversight of these systems. So oversight is always important in any new technology, right? It's like when we started having airplane autopilot, for example, it would be crazy to just say, okay, we have airplane autopilot, just let it fly.

[Jason]

Do we put any oversights over the phase? Facebook and social media companies, they have to be clear.

[Alexander Wang]

They, they do.

[Jason]

They didn't have oversight. The fcc like giving a fine in the review mirror is oversight. It's not oversight. Had no oversight. What regulation is there of AI right now? There's none. You, you're, you're acting under zero regulatory environment right now and China's got a negative regulatory environment.

[Alexander Wang]

It's true that like, so you should.

[Jason]

Be regulated to your emission.

[Alexander Wang]

No, no, no, no, that's not what I'm saying.

[Jason]

Well, wait, wait, you just said that you should be regulated so that we don't have problems. So which isn't.

[Alexander Wang]

I do think that there are a lot of important issues about how we deem what AI systems are appropriate, how we look at what they're supposed to be doing, etc. I do think governing bodies, the US government in particular for example, has to take a deep look, understand the technology, determine what is reasonable, what is not reasonable.

[Jason]

But even in their case, they're looking at the miles driven in the accidents, but they're not looking at the code that you guys are writing. They're not looking anybody's code, they're not looking at the AI systems. They don't even have anybody on staff who could even write an algorithm. Right.

[Alexander Wang]

That's also changing to be clear. Is it in general, do you think.

[Jason]

They'Re looking at any lines of code in, at any of these systems?

[Alexander Wang]

I'm not sure about the answer to that, but I do think they look at a large amount of data. So for example, these.

[Jason]

Okay, they do, yeah, yeah.

[Alexander Wang]

In Europe, for example, there are all these ADAS systems. Right. So there are these driver assistance programs or driver assistance systems and a lot of like high end vehicles that you buy today. Right.

[Jason]

Keep you in the line lane.

[Alexander Wang]

Yeah, exactly. To keep you in the lane. If you have like stopping of traffic, you don't need to do anything.

[Jason]

Yeah. Adaptive cruise control, lane change warning.

[Alexander Wang]

These systems exist, they, people buy these systems, people rely on these systems. And in the eu, for example, where a lot of these car makers are, where BMW, Audi, vw, et cetera are, they have a responsibility to actually both have a large data set they have collected themselves that is able to validate that these systems are performing as well as pass a series of sort of trials and actual.

[Jason]

Oh really?

[Alexander Wang]

Yeah, yeah. Different forms of data that Governing bodies place in front of them.

[Jason]

Well, that would be very interesting. Now to think about it. We do crash tests for cars. You're required to give three cars or something to the government for them to just destroy in their crash tests. But we don't require those cars to go into a lab, get taken over by the governing body and force them to go into real world testing environments. Because there's some real world testing environment where you do self driving up north here.

[Alexander Wang]

I think a lot of these companies buy cheap real estate. They outfit them into like these mini towns so they can create these funny scenarios.

[Jason]

Have you ever been to one of those?

[Alexander Wang]

I've never been, but I've definitely seen.

[Jason]

The video from there.

[Alexander Wang]

The videos?

[Jason]

Yeah, yeah, it's pretty cool. They have like children come darting out, like little cardboard cutouts of children to see if it hits it this way. They can do that in private. But it's interesting that some point the government's going to have to have people who are developers and coders actually getting into the data and understanding something. Some portion of this at the very.

[Alexander Wang]

Least love to create like the driver's test, for example, the driver's license test for, for a self driving car. I mean that will exist.

[Alex]

And I think we've all just become a little bit less concerned about that because when's the last time you heard Jason talk about, I mean, Terminator or.

[Lon Harris]

Like they're going to use the AI to make like a biological weapon that's.

[Alex]

Going to blind you.

[Lon Harris]

Yeah, we are definitely like less concerned about all that today. And I think yeah, Alexander was, he was a bit ahead of the time on that. I enjoy the fact that he is a little bit skeptical of science fiction. I feel like so many of the tech visionaries of the moment love science fiction and obsess over it and try to make science fiction reality. I mean, I think Bezos and Elon Musk are great examples of guys who were like, you know, they grew up reading a lot of this sci fi and a lot of their ideas about space and robots and AI and brain implants are like, I want to bring I about Star Trek and make Star Trek's future happen. And I feel like a lot of what Alexander was saying was more like aware of the sci fi element of it, but a little bit skeptical. Like he's a little bit skeptical about AGI and like we think we're going to get there faster than we do. And he sort of put forth a very fascinating, very sci fi feeling. But he feels the most plausible path to AGI, which is you would create an artificial life form by simulating evolution in a computer. We can't just design the final form. We would design the like the cells or the whatever was in the pond for. And then the computer would like naturally select that up to be like a life form like a human that was sufficiently evolved and like you could tell a guy who's throwing around ideas like that has read his sci fi but hasn't let it. Like I'm going to cloud like this is. I'm going to just try to make what Robert Heinlein was envisioning. Like he's really thinking about it in a practical way and trying to integrate it into what we really know about science and technology. I thought that was fascinating.

[Alex]

If you want more about this, the Lifecycle of Software Objects, a short story by Ted Chiang, which was put together in his latest collection of short stories, is really well worth your time. It's funny how much there is a shared language lawn thanks to our science fiction heritage. And I think think it's, it's fun for me because I've read all the authors that are kind of in question here. So when they make allusions and references it's kind of like my, my, my personal hobby. So that's a real banger for the sci fi nerds.

[Lon Harris]

We should point out Ted Liang also wrote the story that became that movie Arrival.

[Alex]

Yes, Stories of Our Life. But the thing that I want to just double click on before we play this general AI clip is that he talks about how just more compute is not going to get us there. And I think that that's become less of a commonly held view in the let's get the newest Nvidia chips and get 20,000 of them in one building and then push really hard. Meta in particular, Xai in particular have been investing a lot in building up their compute power as a way to create more intelligent systems. It's funny to hear him here say that that's not going to get us all the way to the mountaintop.

[Lon Harris]

What a helpful thing. That made me feel less crazy because I feel like sometimes there is that assumption that we don't need to, to like code things anymore. We just need to put enough chips in a room and then they will figure it out for us. And to me that's always, I'm not a programmer folks, so like, maybe that's how it'll work. But that always felt very counterintuitive to me. Like every other piece of software ever, we, a human had to like figure out how it was going to work. And now we're suddenly coming with this idea that, well, if you just put enough computer compute power together, it'll take care of all of that for us. And I've never really bought that. So it was, it was nice to hear a technologist agree with me in that way, like, oh, I'm not crazy. This is a big assumption for us to make.

[Alex]

It'd be fun to hear what he has to think about this now. But here's Alexander Way you believe in.

[Jason]

General AI that will hit that at some point, AI that is just generally smart can do anything a human can.

[Alexander Wang]

I believe in some sense. I believe in the sense that like for most technological things that humans can conceive of that aren't physically impossible, if humans survive, they'll happen at some point. Like, I think humans are like infinitely creative, infinitely ingenious, etc. I think it's very overblown, the timelines that people are talking about general AI happening. There's a lot of things that are wrong about like the common arguments, one of which is people say that if Moore's law keeps going, then we'll have all this exponential compute. It's only a matter of time before we produce these general AI Moore's laws, It's going to be dead. And then quantum computing is very far away, despite recent releases, et cetera. I think that leg of the argument is not actually that strong. And then I also think it's not even clear that if you have infinite compute, you'll be able to produce general AI. I think that's very unclear.

[Jason]

So infinite compute helps narrow AI because you're doing a number of scenarios and playing out every scenario in go the game. Many more permutations than poker, many more permutations than chess, which is a finite data set. So yeah, more compute power on those things certainly get your you quicker ability. Or even just throwing people into a random video game like OpenAI is sure, definitely. But general AI taking somebody who've mastered chess and then saying master go and then master impressionist painting, it's different.

[Alexander Wang]

It's very different. One of the arguments goes that once you have enough compute, you can create artificial life by basically simulating evolution. It's one of the more vogue arguments.

[Jason]

You have so much compute power that you can say, start with this tiny piece of bacteria, whatever, then grow it and grow and grow an entire evolutionary system to the point at which there is a human like species and then grow that human like species in whatever number of scenarios with a big brain into whatever comes after us.

[Alexander Wang]

Even if you just grow the human like species that's as intelligent as then you're. That's kind of good.

[Jason]

That would be general AI. It's around as well.

[Gu Rao]

Right.

[Jason]

Because general AI would normally most people would define general AI as not even being smarter than us, but being as smart. But somebody would have to code that and program that and build the systems to do that. It's not just going to magically happen.

[Gu Rao]

Right.

[Alexander Wang]

It's very unclear if that's even possible.

[Jason]

That's nr.

[Alexander Wang]

I honestly think that's the most plausible argument. It's very much so science fiction in the sense that we're not close to being able to even validate the hypothesis. So I don't believe in general AI anytime soon.

[Alex]

Lon. I just love how many people we've talked to over the last decade, two decades here on the show. So our archives are deep. We shall mine them. But that's all for twist today. Lon, you're the best. We'll talk to everyone soon. Goodbye. Bye bye.