Guido Appenzeller (2:29)

But there doesn't have to be a knowledge base.

Mat Bornstein (2:31)

Right? It doesn't even have to be a knowledge base. I see, Got it. Okay, so maybe it's just a trained model. It's all the model weights the knowledge, so it's even simpler. So an agent could just be an LLM with a chat interface or something like that by some definition, Right? I think on the other end of the spectrum, there are some people who basically say for something to be a real agent, it has to be something fairly close to AGI.

Yoko Lee (2:52)

Right.

Mat Bornstein (2:53)

It needs to persist over long periods of time. It needs to be able to learn. It needs to have a knowledge base. It needs to work independently on problems. If you take then the most extensive definition, is it fair to say that doesn't work yet?

Guido Appenzeller (3:04)

I think so. It doesn't work yet. Although.

Mat Bornstein (3:08)

Will it ever work?

Guido Appenzeller (3:10)

That's a philosophical question.

Mat Bornstein (3:12)

All right, fair, Very fair. Very fair. So if we take that continuum in between, is there at least a way to chop that up into a couple of categories of software? Maybe degrees of agentic behavior.

Guido Appenzeller (3:26)

And different types of agent? There's some artsy agent that help artists to come up with new bezier curves. There's coding agent, which we like to talk about as the agent of the day. Yeah. Which we use. There's agent that's just a wrapper on top of LLMs.

Mat Bornstein (3:42)

That's right. Yeah.

Yoko Lee (3:44)

I may be the contrarian in this group. All right, look, I kind of think agent is just a word for. For AI applications. Right. Anything that uses AI kind of can be an agent. Now, before. Before we started this talk, I actually went online just to refresh myself about some of the more interesting AI agent perspectives out there. I found a really cool talk from Karpathy that he gave a couple of years ago about agents, which I can describe a little bit, but the really funny part was on the YouTube recommended videos to watch next. It's like AI agents are going to revolutionize your lifestyle and the rise of super intelligent AI. You know, it's just kind of like marketing. And so I actually do think that's what's going on in a lot of ways. The cleanest definition I've seen of an agent is just something that does complex planning and something that interacts with outside systems. The problem with that definition is all LLMs now do both of those things, right? They have built in planning in many cases and they at least consume information, you know, at least from the Internet, maybe from some servers that expose information through MCP or some other protocol. So the line really is very blurry. And what was so interesting about the Karpathy talk is he basically he related to autonomous vehicles and said AI agents are a real problem, but it's like a 10 year problem, it's like a decade problem that we need to work on. And I think most of what we're seeing in the market now is not the decade version of this problem. It's like the weekend demo version of this problem. And this is why we sort of generate so much confusion. You have this kind of poorly defined nebulous thing that LLMs are kind of consuming themselves over time. And so I don't think anything we have are actually agents. And agent itself may be a poorly defined and kind of overloaded term. But if someone's willing to do the hard work and define exactly what it's like to kind of be a human, but in digital form and spend 10 years to make it actually work, you know that that's what I'm excited to see.

Derek Harris (5:33)

Okay, so defining agents is a difficult job. Maybe it's easier to talk about how people use the tools they call agents and what are the differing degrees of agentic behavior.

Guido Appenzeller (5:42)

I wonder if part of the conversation is redefining agent because we all know that agent as a term just not a great. It means so many things to so many people. If it's interesting to dissect, like what do we mean? What do different people mean when they say agents? What are different ways we could utilize this process we call agents.

Mat Bornstein (6:01)

So it seems to me if we're trying to define agents or maybe even degrees of agentic behavior, which might be a little easier, there's something like a user interface aspect to it, right? Where something that's a pure copilot where basically user goes back and forth with an LLM to work on a particular task that's often not called an agent. Is that fair? There's a little bit. The copilots versus agents UI models.

Guido Appenzeller (6:26)

Yeah, I guess. What are the elements we would think that goes into agentic behavior? Like Matt mentioned, planning could be one. There could be decisions made by the agent. There has to be LLM somewhere. But curious about your take.

Mat Bornstein (6:41)

I think another definition we heard from Anthropic recently was this idea that an agent is an LLM running in a loop with tool use. Right? Which there's two important parts of it. One is this notion that it's not just a single prompt and not even just a single static sequence of prompts, but something where the LLM takes the output of a prompt, feeds it back into itself and based on that makes decisions on what the next prompt and likely also went to abort like you went to complete a task. I think that for the real agents or the more agentic behaviors, I think that's a reasonably good definition. I think the other thing about that.

Yoko Lee (7:17)

But just by that definition, isn't every chatbot effectively an agent then in this world? Right. Like if I go just to chatgpt.com and use their latest reasoning model with web search, Right. Isn't it using tools and feeding its outputs into a new prompt in order to do kind of chain of thought?

Mat Bornstein (7:36)

Chain of thought is a little bit in between. If it's just a single prompt that comes back with a result, then it wouldn't have this notion of planning and doing a more long term concept and deciding itself when it is complete. Right. If you have a chain of thought reasoning where I'm giving a more complex task that's starting to look. I agree.

Yoko Lee (7:51)

I just think it's really tough to define a system based on what someone says to it. Right. Because these are by design unstructured inputs. These systems will accept literally anything. And so sure, if you tell it, you know, what's today's weather, I would agree that's not agentic.

Derek Harris (8:08)

Right.

Yoko Lee (8:08)

That's just fetching from an API. If you ask it define a new philosophy of weather, right. It'll happily go do it. So. So it's like an agent if you ask it one thing, but not an agent if you ask it another thing. I think that's a lot of the confusion in the market around this. And if we spoke in the terms that you're talking about, Guido of like, hey, this is an LLM in a loop with a tool like that's actually a much more productive way to talk about it, I think.

Mat Bornstein (8:28)

Yeah, yeah. I mean that said, it seems like we're seeing some degree a specialization of user interfaces in sort of two directions, right. There's let's say a cursor or something like that, which really emphasizes the tight loop between the user, the tight feedback loop between the user and the LLM and the thing I'm working on, right. So I want immediate gratification when I do something and sort of response time matters. Then there's more. The backend as source code management System type plugins where more about throwing something over the wall by maybe answering a couple of questions and then you try to maximize the amount of time the agent can work independently. So it seems like I think you're right that there's no clean system definition split between the two, but there seems to be a little bit of an user interface specialization. Is that a fair statement?

Guido Appenzeller (9:13)

It almost feels like for all the use cases we've described, there's one element that all agents have which is reasoning and decision. Would you call just a call to LLM to say, translate this text to JSON? That's probably not the agent. But then if you ask LLM to say, hey, decide where you know this response goes and route it, for me it feels more like an agent than before. So it almost felt like planning. I'm actually not sure. Does the agent need to plan or does it need to decide? Maybe both. I actually feel like it's a multi step LLM chain with a decision tree.

Mat Bornstein (9:49)

A dynamic decision. A dynamic decision, yeah, I think that's fair.

Yoko Lee (9:52)

I think we've all just been nerd sniped. I just think it's like humanities. People love classifying and they draw kind of like fine distinctions between different types of things, entities, whatever. We're computer scientists like for, you know, you know, not there's anything wrong with humanities, but we're just not that. So I think we're not well equipped when it's a bit, isn't just 0 or 1, it's maybe something in between. And we just talk about it a lot. We like try to like coerce it to one value or the other.

Derek Harris (10:18)

Yeah, of course. Agents are more than pure technology. They're also becoming products, which means they need to be marketed. And how someone positions their product has a major effect on how they price it. What's more, the ultimate value of any given agent, which is still to be determined for the vast majority of them, is to what degree they can actually replace or simply augment.

Mat Bornstein (10:41)

Human workers raise an interesting point, which is I think that is a marketing angle to agents. Right. I've heard this narrative from a couple of startups that they're basically saying, like, hey, we can price the software that we're building much, much higher because this is an agent. So we can go to a company and say, you're replacing a human worker with this agent. The human worker makes, I don't know, $50,000 a year, and therefore this agent you can get for only $30,000 a year. This sounds really compelling from a first glance. And actually, I mean, there's some value to it in the very early days because it essentially, it's very easy to understand comparative pricing for somebody who has to make a buying decision right now. On the flip side, we all know that the cost of a product over time converges towards the marginal cost of production. Right? And so today, if I used to use a translator, maybe to translate a page of text, today I use ChatGPT. I do not pay ChatGPT like I paid my translator. I paid a tiny fraction of a cent via the API, which is the actual cost. So I sort of wonder how much of the agent debate is driven by marketing and pricing.

Yoko Lee (11:40)

I just actually think this is a really interesting topic. What fields can you think of that are actually suffering complete replacement from AI or AI agent? And this is a setup. I'll warn you, I have another extreme point of view that I'll say afterward. But can you think of fields where this is actually happening?

Guido Appenzeller (11:56)

Not completely, but definitely partially, because there's a lot of, for example, voice agents, I replace receptionists, people who would get back to customers. So there's definitely a lot of workloads that have been offloaded from the folks who traditionally did the job. But I don't think they're 100% replaced. They can do something else. But we are seeing headcount growth in some areas are slowing. So it's not that existing jobs are being replaced. It's more like they're hiring net new humans slower.

Mat Bornstein (12:28)

I think it's exactly right. I mean, I think in few cases humans will get replaced by AI. In most cases, two humans will get replaced, one human by one human. That's more productive with AI.

Yoko Lee (12:39)

Yeah. Or maybe they keep the two employees.

Mat Bornstein (12:42)

Maybe they go to three employees because now they're more productive.

Yoko Lee (12:44)

Yeah, right, right. It's just a really interesting question. And the reason I think it's really relevant to agents is I think part of the ethos and part of the confusion around agents is this idea that we actually will develop human replacements and that this thing we called an agent, which by the way, is a name for a person. Before we had AI, we had people called agents. And we still have all kinds of people called agents. And it just doesn't seem like that's happening. Right. Not in the replacement sense you mentioned, Yoko. With agents, we've already always had, you know, customer support, Automation. We've had 1, 800 numbers where, like press 1 for sales plus, you know, if that's existed for a long time, this is a much better form of that. Obviously translation is a great example too. Guido. These systems can perform translation extremely well. But you're probably not going to just stick something to ChatGPT and then publish it on your website. There is actually work that needs to take place. And I think the reason for this is there's just fundamental creative work in most things that humans do. I think from our kind of perch in Silicon Valley we can forget that sometimes that people all over the country and doing all sorts of jobs actually have hard jobs. And not just hard in the sense of someone's got to do it jobs, but hard in the sense of it does take thinking and human decision making, which I just don't know. That AI kind of has what we would think of as decision making or intent. It's a system that still somebody has to push the button, right? It may be running somewhere, it may do a great job or whatever. Someone tells us to give it a prompt and hit go. And to me that's a lot of the, that's a lot of the confusion around agents. We're all thinking at some point a human person with intent and creativity and thinking is going to be replaced. I'm just not sure that even is theoretically possible. It's almost just like a catch 22 to say an AI system is thinking for itself because somebody has to have sort of created it. You know, this is old sci fi philosophy I'm getting into now. But like I actually do think it's a part, a big reason for the confusion that, you know, we sort of experience now.

Guido Appenzeller (14:27)

It's interesting because there's two types of agent we're already talking about. There's one type where the agent is replacing humans work with humans. Do you think humans can do? There's the other type of agents that's more low level system processes. They work with each other, they hand off tasks to each other. To some extent, agents are like technical details in the system in that way. But we mean both when we talk about agents.

Mat Bornstein (14:50)

In that case, is there actually a difference between an agent and a function?

Guido Appenzeller (14:54)

I think so. I think agent will be multiple functions with LLMs in the middle.

Mat Bornstein (14:58)

If I have a low level agent and I'm giving this low level agent a task and I get back a task result, it looks a little bit like a classic API call, but with.

Guido Appenzeller (15:07)

The LM in the middle to make decisions on what to do for that API call.

Mat Bornstein (15:10)

So. But I understood. But that's sort of how this function works internally to some degree.

Guido Appenzeller (15:15)

Yes.

Mat Bornstein (15:15)

Right, yeah. So from the outside, would I care.

Guido Appenzeller (15:18)

You wouldn't care. It's like most of the time when we see AI SDRs, when we talk about AI SDR agents, what we mean by that is when the agent can go to the CRM, pull something out and then filter the list, draft an email and send the email. So that feels very process level instead of human level. Yeah, so that's what I meant.

Mat Bornstein (15:42)

If you don't know how this thing works internally, a classic function and an agent become indistinguishable.

Guido Appenzeller (15:48)

Totally, I absolutely agree. But when you as a programmer, when you write the function, you will define agent. That's this interface.

Derek Harris (15:58)

We'll get back to pricing shortly, but first let's dive a little deeper into this discussion of how interacting with an agent is different than or similar to traditional software based functions.

Yoko Lee (16:09)

So here's one interesting thing to think about on that topic. I totally agree with you Guido, and I think you sort of agree too. There's that it's really a function. If you kind of just look at it that way. Shareable, reproducible functions have never really been a thing. This has been one of these long time goals that people in the market have tried to say, oh, I can just write a function and then anybody on earth can use it, right? Like, you know, we have packages that you can download, download a whole package with various functionality, but literally just one function that you can share if you kind of squint a little bit. That kind of exists now with AI because you have these models that's trained by somebody, somebody else may download it, fine, tune it, train a Lora, package it up into some new and interesting way and then it's actually immediately available for someone else to use on hosting services or hugging face or something like that. So while it does seem to be just an implementation detail whether you're using an LLM or not, there is this interesting thing where the model itself takes up so much of that functionality in the function and it's just a different kind of animal compared to normal code. It's actually more, it's kind of shared by default in a way because nobody's going in and training their own model every time they're writing code. You know, it's obviously heavy, it's harder to move around. There are all these different characteristics from normal functions that some of which are actually very desirable, some are kind of bad characteristics you don't want, but many of them are kind of interesting. And I think we'll actually see new infrastructure, new dev tools kind of built around this in the Long run, I.

Mat Bornstein (17:26)

Think what makes sense, I mean, when, if we go back in time, the last time we sort of invented a major new component for building systems, it was probably networking, right? How we thought about calling a function before networking afterwards changed a lot, right?

Yoko Lee (17:40)

Totally, totally.

Mat Bornstein (17:41)

The complexities of APIs and the infrastructure around it is completely different today.

Guido Appenzeller (17:45)

This is such a good point because now I think about it, I feel like humans are just functions too. Like if you have a thought experiment and then replace LLMs in the program to a human, like how the kind of answers we'll give to the program is not that different from what the LLM will give to the program.

Yoko Lee (18:02)

So if we actually all get hooked up to servers one day and can be called as a function from lambda, then I will agree that agents have been created.

Mat Bornstein (18:11)

That's what an agent is, isn't Mechanical Turk. Exactly that or maybe even your email inbox, I don't know.

Guido Appenzeller (18:17)

There's a Amazon.

Yoko Lee (18:18)

Sounds like an agent to me.

Guido Appenzeller (18:18)

Supermarket. A while back in Soma, I think they were advertising that it's computer vision models behind the scenes identifying what you took from the supermarket. But then people found that they hire a lot of people behind the scenes to actually label the data in real time. So the humans in that case are the functions that today maybe secret agents, right. Replaced by LLMs with.

Yoko Lee (18:42)

Well, but this was exactly my point though, right? There actually is important creative work. Even in a grocery store checkout clerk, you could naively think, oh, this is an easy job. Actually it's not an easy job at all. Right. And so you can take this work and kind of shift it and you can squeeze it down with automation and stuff, but it never really goes away.

Guido Appenzeller (19:00)

Oh yeah, absolutely.

Yoko Lee (19:01)

Yeah.

Derek Harris (19:02)

All right, so given all of this, how should companies think about pricing their agents? Per seat, per token, per task? Hint, it might be too early to truly tell.

Mat Bornstein (19:13)

Usually if you introduce a brand new product category, you often initially put a pricing the price is against the status quo, right. Whatever you replace or augment in some cases. But let's assume we have a direct replacement, right. So that's I think where this idea from. Oh, this replaces a human, which it doesn't. And but if it would, then you could charge, you know, x amount for it. And usually over time competition kicks in and you're effectively priced by how much your competitors are charging. You start sort of an erosion. Then it depends on many things like how much of moat do we have? Do you have customer lock in? Right. And so on long term converge against the marginal Cost of production, which, I mean, look, if I look at most agents today, it's probably very low, right? Any agent you can purely model in software with a couple of LLMs calls, you can run at a very low cost and the cost is decreasing over time.

Yoko Lee (20:05)

And I would sort of argue that's kind of already what's happening, that in practice, most AI applications, and in particular if we want to call them AI agent applications, you know, they have their sales pitch around, you should pay us X because we're saving you. You know, it's like a classic ROI calculation.

Mat Bornstein (20:23)

Value.

Yoko Lee (20:24)

Yeah, exactly. Yeah, exactly. Value based pricing. But in practice, I think most buyers are actually pretty sophisticated about what's going on under the hood. And to your point, they know it's pretty simple stuff happening. And so it's like, hey, what does it cost you to run all these GPUs and we'll pay you some premium over that. And I think that's how a lot of vendors are pricing in practice these days.

Mat Bornstein (20:42)

I mean, long term you'd expect pretty healthy margins. Just like in SaaS, right? Which software traditionally has very good margins.

Guido Appenzeller (20:48)

It's so funny because we always advise companies to not price based on the margin, but price based on the value you add, whatever that could be. It could be compared to other vendors on the market, could be compared to just, you know, what it is building in house. And traditionally for infra, a rule of thumb not always the case is that if the Surface is used by a human, it's a per seat pricing and if it's a service, it's used by other machines, it's a usage based pricing. And I actually don't know where to put agents here.

Mat Bornstein (21:20)

It could be used by either. Right? A human could be using an agent.

Yoko Lee (21:24)

Look, I think your analysis is exactly right. And the reality is most AI companies don't know what value they're generating yet. This is so new and so nascent that it's like, hey, we're just going to charge something that we're not going to lose money on. And you know, in the case of OpenAI, they have how many millions of users, they probably don't have a very strong sense of what they're all using it for. And once they do, and you see this more, they're trying to verticalize a bit more and have kind of specific products for specific use cases, code obviously being the big one, then you'll be able to see the pricing kind of catch up is kind of my hypothesis.

Guido Appenzeller (21:55)

This reminds me, the opening I point you Brought up, I was thinking about AI companions because that's the closest to per seat. Per seat human pricing. Like you can't charge someone every sentence they talk to their companion. Although some of the foundational models there.

Yoko Lee (22:12)

Are services that will charge you per response. I haven't used them, but they do exist.

Guido Appenzeller (22:17)

Yeah, I see, wow. Okay, so usually it's kind of weird to charge someone like buy tokens of how much they talk to the companion. Other than like a flat monthly fee.

Mat Bornstein (22:28)

It doesn't feel like a true friend if they charge them.

Guido Appenzeller (22:30)

Right, exactly. It's very transactional.

Yoko Lee (22:33)

This is look is all theory, right? People love sitting around and talking. Oh, we're going to charge per person, per task, per world economy that we rescue. It's like it's all made up, right? I think Guido's thing was exactly right. Let's look at the actual technology underlying what we're calling agents right now. Where are they being deployed and why? And honestly, the pricing, the marketing, the sales tactics, all of this kind of follows from what they're actually selling.

Derek Harris (22:55)

If I'm selling something that looks like an agent, but I haven't truly figured out the value I'm providing to my users, how do I justify the jump to a higher price point when I do figure out that value?

Yoko Lee (23:05)

You just need to be selling a solution rather than a product. This is really well worn expertise in enterprise go to market code. You can somewhat see the decoupling of price from the underlying technology now because it really works. There's very clear ROI to people who use it. And so as a VP of Engineering or a cto, you can look at this and say, okay, I'm actually saving a lot of money and my guys are getting a lot more productive. I can value, I can do a.

Mat Bornstein (23:28)

Normal job is they're happier.

Yoko Lee (23:30)

Yeah. So you're kind of buying a solution, right? You're buying from a vendor, something that solves a problem for you, which again Microsoft, Oracle, Salesforce people have been doing forever. Once we start to see more of that, it's going to be these things that become real products and kind of decouple pricing and look kind of like real businesses.

Guido Appenzeller (23:42)

I think, I think it's dictated by the high level application. So I'll give you an example. So I'm a Pokemon Go player. So for those who have played Pokemon Go, once you collect enough Pokemons, you are out of storage in your pocket. So you need to pay extra to buy a new bag, virtual bag that you can put more Pokemon in. And as an infrastructure Investor. I invest in storage businesses and then when I look at how much I need to pay for like 30 extra Pokemon, it was thousands of times more expensive than what storage is. So it actually reminded me.

Mat Bornstein (24:17)

I'm surprised it's only thousands. What if it gets 10 to the price?

Guido Appenzeller (24:21)

There's a whole price curve on Pokemon storage, it turns out because this is.

Yoko Lee (24:25)

One JSON blob basically. I know, can they charge you like $5?

Guido Appenzeller (24:31)

And then the normal Pokemon players, they wouldn't think about this like how much do storage cost? Right. Like a normal Pokemon player will be like oh, this capability, I would be happily paying thousands or more than I were to have a S3 bucket somewhere. So one of it is monopoly. So it's an application layer monopoly that you wouldn't have been able to store the Pokemon anywhere else. And two, it's a use cases, it's for a different audience. It wouldn't be asking these questions, it would be thinking about what is the net new value? What's the net new cost? I will be willing to foot the bill for if I were to get this value. Is it a fun game? It's a fun game. Take a hundred more dollars.

Yoko Lee (25:09)

Yeah, I think that's exactly right. And implicit is what you're saying is this idea that the product or the solution has to actually work for them. Right. For a less technical person who's, you know, the person who's not going to try to provision their own storage bucket.

Mat Bornstein (25:22)

To self hosted monster for Pokemon. Yeah.

Guido Appenzeller (25:25)

And it's quite defensible differentiated too because Pokemon Go is not open source. There's no other replacement of Pokemon Go. There's only one Pokemon Go. So there's only one place where you would be willing to pay so much money for Pokemon storage.

Mat Bornstein (25:39)

Plus very strong brand plus you have a little bit of network effect because you can play together.

Yoko Lee (25:43)

Yeah.

Guido Appenzeller (25:43)

And then we'll see the AI agent version of this. I can't wait to see the AI companion version of this plain storage for AI companions wardrobe.

Derek Harris (25:53)

As the AI market continues to shake out and evolve, where will agent capabilities ultimately live? For example, can they live inside LLMs or must they call external tools? And who's ultimately in the best position to influence this?

Mat Bornstein (26:07)

Super interesting question. Right. What's the system's perspective of how an agent is built? And I personally think that architecturally there really is no difference between your typical SaaS software today in agent in terms of how you build it. And let me explain why. Right. So in agent we said you have sort of an overall Loop with an LLM and prompts that feeds into itself plus external tool use. The LLM itself, you probably want to run a separate infrastructure just because it's highly specialized. You need these vast GPU farms can't easily run today's large LLMs in a single GPU. So it's a very specialized infrastructure that's externally. So the LLM call is external, the state management. Well, today in SaaS applications, we do all the state management externally in databases or something like that. So you probably also want to externalize that, right? And then what remains is fairly lightweight logic where I'm basically I'm taking contexts that I retrieve somehow from databases, I assemble that into a prompt, I run the prompt, and then I occasionally invoke tools. Maybe I do that with MCP or something like that with an external server. But the core loop is actually pretty lightweight and I can run a gazillion agents on a single server. Not a gazillion, but many agents on a single server. I don't need a lot of compute performance for that. Does that sound about right?

Yoko Lee (27:19)

Yeah, yeah, I totally agree. The interesting architectural question for me has always been how do you handle the kind of non determinism that may come. Many of these successful AI applications that we all use and love really just spit model outputs back out to the user, right? Like a chatbot or image generator. It's like, hey, I called the LLM, here's what I got, you know, good luck. When you try to actually incorporate the output from an LLM into the control flow of your program, that is actually a very hard, very unsolved problem that you know, to your point, there are relatively minor architectural differences today, but this may actually drive more significant changes in the future.

Guido Appenzeller (27:55)

I actually think the winners will be the specialists, not the foundational models. It's the people who will build on top of the foundational models or fine tune the foundational models. So like a very artistic example of this is that I've been spending the last two weeks just prompting GPT4O their image model. It's very good at cartooning, so it's very good at manga. It can spell, so it has a storyline. But then I realized that there's only top two or three styles. It's good at. So it's good at Ghibli, it's good at manga. And then there's variations of the style in that realm. So now where art comes in is that the market likes out of distribution art. Everyone doesn't want to see the same things. Over and over again. Because that's how they value art. Something that's different.

Yoko Lee (28:41)

Ideally, maybe. Ideally, maybe.

Mat Bornstein (28:43)

Did in some reason define art as out of distribution? Samples.

Yoko Lee (28:47)

Yeah.

Guido Appenzeller (28:49)

Art can be in distribution. That's pop art, right. It could also be out of distribution. That's like when Impressionism came up many years ago, everyone was drawing Impressionism. And at the time, the painters before they were like, what's wrong with your eyes? Why are you drawing blurry images? So styles come and go, but because of that, I think it's a pushing distribution question. The foundational model will never cover 100% of everything. So it's really up to the humans and specialists of the next wave to come up with the new data, new workflows, new aesthetics, to push that distribution.

Derek Harris (29:25)

Of course, at the end of the day, agents are only as useful as the tools and data to which they have access. So what happens if major web platforms decide they want to keep agents from accessing their data?

Mat Bornstein (29:36)

It seems like one of the hardest things about agents today are data modes. In some cases, just because they're technically difficult, I'm trying to access data, an agent is trying to access data, and it's just very hard to integrate with that system. In some cases, it's very deliberate. Right. My iPhone, the photos are not accessible via any API because it's a walled garden.

Yoko Lee (29:54)

So sort of data silos.

Mat Bornstein (29:55)

You're talking about data silos, right? So is that something that's holding back agents or is making them more difficult? Or to make it even stronger? Consumer companies traditionally often were opposed to offering automated access to their services because they want their user engagement, they want the time to advertise to the user. Will that limit how much we can.

Guido Appenzeller (30:16)

Deploy agents, and would that be changed once we have the browser? Native agents that can browse the web and browse.

Mat Bornstein (30:22)

Great question. Yes.

Yoko Lee (30:23)

Yeah, I think that. I think Yoko is totally right. You know, it's like there's strong incentives for people who own data about, you know, physical entities, people, businesses, et cetera, to keep it to themselves. Right? Especially because they may be scared what AI is going to do to them, by the way. So they're kind of clinging tight to what they have. And these problems are rarely solved by defining a new protocol and just saying, hey, if we make it easy for people to give away their core assets, they'll just do it. Obviously, that's very unlikely to work, but someone eventually will solve this by saying, hey, if your data is publicly visible, we're going to get it. You know, it's like by the way, it's not actually your data, it's data about so why should you be holding onto it?

Guido Appenzeller (31:02)

Actually I feel like the new advancement in models may just change the data mode kind of to the point of today web browsing using an agent doesn't work super well. It's very slow, it's very clunky. You have to try it multiple times for it to do any task. But imagine if we have foundational model capability of giving an agent ability to go to any website logging as a human. We'll table that one. I don't know how agent identity works yet. Or go ssh into a server like execute certain commands or spin up a virtual machine for a mobile or access a device in a device farm to play Pokemon Go. Maybe those are the data traditionally only available to humans under that account now may be available to agents.

Mat Bornstein (31:48)

There's also the opposite that could happen, right that basically all the consumer sites are starting with more and more complex anti agent captchas trying to keep out the agents because they only want the humans that have attention to come to those sites. I mean I recently did use one of these deep research tools, one of the major LLMs and one of the steps, if you look through it, all the steps I went through was like trying to see how I can get around a capture mechanism for a site. There was an actual reasoning step, right where basically it know what information it wanted and it was blocked from accessing it. So is that how dystopian is the future going to be here? It solves it actually.

Yoko Lee (32:23)

I mean it's so interesting. So here's a really early machine learning example of this. I don't know if you guys remember when Gmail first implemented ads it was a big controversy because they basically said okay, we are not going to read your emails, but our algorithms are going to read your emails and we're going to suggest ads that you should watch, you know, or click on based on that. We all sort of, I think just forgot and got used to it. I still think we don't love the idea, but we kind of live with it. But some of the data providers reacted by removing data from email. So Amazon famously now when you order something, they send you a confirmation email that says hey, you just ordered something. Click here to find out what you ordered, when it's going to arrive or any information you might want to know. And so that actually did happen in practice in that example that the major data holders kind of found ways to withhold it. It'll be Interesting to see whether that's possible now or not.

Guido Appenzeller (33:10)

But that same data is scraped on the client side from the ad networks I install.

Yoko Lee (33:16)

Oh sure, yeah, yeah, yeah, yeah. There's always some other way. Yeah, not maybe exactly the same, but pretty good prox. Yeah, yeah.

Mat Bornstein (33:23)

It may be that it's much harder to tell the difference between an LLM and a human than a classic, you.

Yoko Lee (33:29)

Know, so yeah, yeah.

Mat Bornstein (33:30)

API call mechanism and a human. That may be, that may change the dynamics.

Derek Harris (33:34)

Finally, Guido, Matt and Yoko answer an obvious question. On the longest timeline into which we might have clear visibility, what needs to happen to make agents a truly game changing innovation within the next say two years?

Mat Bornstein (33:46)

I think the positive vision is that in two years we figured out how an agent working on my behalf can use most of the tools that I have access to. I think it's also clear what are all the pieces that are missing for that. Right? We have not figured out security authentication, access control for agents working on my behalf yet. We have not figured out how data retention works. We have not figured out the relationship with consumer websites that potentially want to block that agent. But if you had that, it could make many tasks much, much easier. Right. Today if I have data sitting say my Google Drive or so, right? How easy I can reason about that data versus other data that's in more fragmented sources. It makes an incredible difference. So I think that that's the bull case, right? Where you have agents that can take all the data that you can access, they can access on your behalf and perform tasks on your behalf and save you a ton of time. It could make you, depending what you do, like you know, I'm multiple times as productive as you are today.

Guido Appenzeller (34:41)

My answer to that is actually different modalities on the foundational model. Today it's still very much text based and that worked really well for co and text based tests. But then for more visual first tests, there's just no one to one mapping. Even for web browsing, it's like a very clunky experience of take screenshot every couple seconds and send it back to the foundational model. So I will actually bet on multimodality when it comes to if we train the model with different traces of clicking on buttons on the website, navigating the web using different devices, drawing, producing vector art, I think there will be net new things that the model could unlock.

Yoko Lee (35:19)

On the agent level, you can probably guess my answer. If we don't use the word agent two years from now or five years from now, I think that's a huge win. There's actually a fun paper put out by some folks at Columbia, I think, called AI is Normal Technology, and they sort of make the argument that there's a false dichotomy out there. It's like AI is either going to bring about utopia or dystopia, meaning everything's going to be amazing because we have AI, or everything's going to be terrible. This is kind of the national discourse, but if you just think of it as normal, like water or electricity or the Internet or things like that, I think that's the world we're kind of headed towards. An agent is this kind of way to help us get there. And so that's my goal. I mean, this stuff is just incredibly powerful, and we understand how to use it, we understand the use cases, and we're kind of, you know, we're kind of putting it to use for us.

Derek Harris (36:05)

And there you have it. A thoughtful and Insightful look at AI agents from the ACE16Z8 for team. If you enjoy this episode, please do rate and review the podcast and share it far and wide among your friends and colleagues. And stay tuned for more great episodes in the weeks to come.