Sam Altman (3:07)

So, yeah, is that a book?

Dharmesh Shah (3:09)

I think it's an average. I think the average book is like 240,000 words, I think, but I'm not sure.

Sam Altman (3:13)

That's not a lot. So the way that I use ChatGPT is like, let's say a fun way is I'll, I'll put a historical book that I loved reading and I'll be like, summarize this so I remember the details. So you're telling me that if it's a thousand page book, it's not even going to accurately summarize that book?

Dharmesh Shah (3:29)

It won't fit. Like, if you paste something large enough into ChatGPT or whatever AI application you're using, it will come back and say, sorry, that doesn't fit. Effectively, what they're saying is that does not fit in the context window. So you're going to have to do something different.

Sam Altman (3:44)

All right, A few episodes ago, I talked about something and I got thousands of messages asking me to go deeper into explain. And that's what I'm about to do. So I told you guys how I use ChatGPT as a life coach or a thought partner. And what I did was I uploaded all types of amazing information. So I uploaded my personal finances, my net worth, my goals, different books that I like, issues going on in my personal life and businesses. I uploaded so much information and, and so the output is that I have this GPT that I can ask questions that I'm having issues with in my life, like how should I respond to this email? What's the right decision? Knowing that, you know, my goals for the future, things like that. And so I worked with HubSpot to put together a step by step process showing the audience, showing you the software that I used to make this, the information that I had chatgpt ask me all this stuff. So it's super easy for you to use. And like I said, I use this like 10 or 20 times a day. It's literally changed my life. And so if you want that it's free, there's a link below, just click it, enter your email and we will send you everything you need to know to set this up in just about 20 minutes and I'll show you how I use it again 10 to 20 times a day. All right, so check it out. The link is below in the description, back to the episode. I usually use projects and I have like, let's say a health project and I'll upload tons and tons of books or tons of blood, blood work and I hope, I'm hoping that it's going to pull from all those books in my project. Is that true?

Dharmesh Shah (5:10)

That that is true. So here, and, and this is a perfect segue, right? Because this is the next big unlock. So the number one thing to like understand in our heads is there's this thing called a context window. Here's why it matters. So let's, we're going to take a, we're going to pop that on the stack and we're going to push down the stack and we're going to come back to it. So the thing we have to remember is two things. Number one, it doesn't know what it's never been trained on. That's one of the limitations, right? So if you ask it something that only you Sam, have in your, in your files, in your email, wherever that the training model was, I mean the LLM was never trained on, it's not going to know those things. Doesn't matter how smart it is. It's just information it's never seen. So it's not going to know that. That's kind of problem number one. Problem number two. So let's say your website for Hampton was actually on in the training side, right? Because it's on the public Internet or whatever. But the training happened at a particular point in time. Like they ran the training, ran the training Ran the training and said, okay, we're done with the training now the machine is done. Let's let the customers in right now. If the website changes, it's not going to know about those new updates that you've made to your website because the training was done at a particular date. If completed, it's kind of training course. So those are two things we sort of have to remember, is that it doesn't know what it doesn't know, and number two, that the things it did know were frozen at that particular point in time. Right? So it hasn't seen new information. And those are relatively large limitations. Right? So especially if you're going to use it for business use or personal, it's like, well, I've got a bunch of stuff that I want it to be able to answer questions about or whatever inside my company or inside my own personal life. How do I get it to do that? And so here's the hack. And this was a brilliant, brilliant discovery. So what they figured out is to say, okay, let's say you have a hundred thousand documents that we're never on the Internet. That's in your company. It's all your employee hiring practices, your model. Here's how we do compensation, all of it, right? It's like all of you have 100,000 documents. And obviously you can't ask questions about those 100,000 documents straight to ChatGPT, doesn't know anything about those, Never seen those documents. So this is, and we talked about this two episodes ago, this thing called vector embeddings and rag retrieval, augmented generation. And I recommend you folks go listen to that. I think it's a fun episode, but I'll kind of summarize it, which is what you can do. And what we do is to say we're going to take those 100,000 documents and we're going to put them in this special database called a vector store. A vector database. And what we can do now is when someone asks a question, we can go to the vector store, not the LLM, go to the vector store and say, give me the five documents out of the hundred thousand that are most likely to answer this question based on the meaning of the question, not keywords, based on the actual meaning of the question. So it's called a semantic search is what the vector store is doing. So it comes back with five documents, let's just say. Now, as it turns out, five documents do fit inside the context window. So effectively we said, okay, well yeah, it would have been nice had you trained on the 100,000 documents. But that was not practical because I didn't want to expose all of that. I'm going to give you the five documents that you actually need. I'm just going to give it to you in the context window. And now, as you can imagine, it does an exceptionally good job at answering the question when it knows the five documents that should be looking. You just gave it to them. Right. So it's having this. So we'll kind of jump metaphors here. It's like hiring a really, really good intern that has a PhD in everything. Right. They went to school, they read all the things, read all the Internet. The intern knows everything about everything that ever was publicly accessible. They're trained, show up for the first day of work. That's all they know. They're not learning anything new and they know nothing about your business. Now it's like, okay, well, I know you know everything about everything. I have this question about my business. Here are five documents that you can read right now and answer my question. It's like, oh, I can do that.

Sean Puri (8:51)

I like that analogy. The intern with the PhD and everything. That's so much how it is. Right. It's as helpful and available as an intern, but it's as knowledgeable as somebody with a PhD in everything.

Dharmesh Shah (9:04)

Yes.

Sean Puri (9:04)

And then like you said, you know, my. Another analogy for that is like, it's a store, you have shelf space, which is kind of limited, but they do have a back and you can always get. Send the employee to the back and see if they can find it in the back for you.

Dharmesh Shah (9:15)

Right.

Sean Puri (9:15)

That's kind of like what you're saying. Put it in the database, they can go fetch the specific thing that you're asking for because, you know, you gave it access to the back, you gave it a badge that lets it go in there.

Sam Altman (9:24)

Have you uploaded all of, like, have you figured, first of all, I want to know what your chat GPT looks like. I want to know how you use it on a. Like, I just want you to just screen share. Just like show me exactly what you do. But also, have you uploaded your entire life, like, have you uploaded all of HubSpot to ChatGPT where you could just ask it any question?

Dharmesh Shah (9:42)

Yeah. Multiple times. Right.

Sam Altman (9:44)

So, and what format. Tell me how you did that.

Dharmesh Shah (9:47)

So I did. So OpenAI has called this, it's called an embeddings algorithm that takes any piece of text, a document, an email, whatever it happens to be, and creates this kind of point in high dimensional space called a vector embedding. And you know, a point in high dimensional space. So in three dimensional space, physical space that we know of, we think of points being in three dimensions, X, Y and Z axis. Like, oh, here's where this point is in space. High dimensional space, you can have 100 dimensions, you can have a thousand dimensions and describe each document as this kind of point in space. So what I've done. So it used to be in the early kind of GPT world, the number of dimensions you had access to was roughly like 100 to 200 dimensions. And so you would lose a lot of the meaning of a document, right? They would sort of get it right, it would sort of capture the meaning. And then, then we went to like a thousand dimensions. It's like, oh, well now it can much more accurately sort of represent and capture a document of kind of arbitrary length and be able to find it, give it a prompt or given some sort of search query. And then recently, within the last year we've gone, the latest algorithm from OpenAI embeddings algorithm is like 3072, I think, dimensions.

Sean Puri (10:55)

But where do you do this? Do you just literally upload it as a project or you had to do an API connection. How do you actually do this?

Dharmesh Shah (11:02)

I'm doing an API connection, right. In fact, I'm running the least, Let me see where it is now.

Sean Puri (11:07)

And anyone could do this or you have special access because you're friends.

Dharmesh Shah (11:10)

No, anyone can do this. The API for the embeddings model, they have two versions. They have the 3000 dimension version, they have a 1000 dimension version.

Sam Altman (11:18)

And it's the results of this. Like, are you driving a nascar and I'm driving like a scooter? Like, is that the difference? Like, if I just like, for example, what I will do is I'll just download my company's financials and I'll upload it and then I'll explain what my company does. But the way that you do it is a lot different. Now, are we talking a massive gap in results that you get versus what I get?

Dharmesh Shah (11:41)

Yes, the short answer is yes. And the reason is so I do that as well. In terms of how I'll describe the company or whatever, I try to provide it context and that's why it's called the context windows. You're trying to provide the LLM context for what you're asking it to do. The difference is that, you know, because I can go through like, and by the way, the richest, and I'm working on a kind of nice and weekends project right now. That takes email, which you know. So you would be amazed, like, if you had to write no other words right now, if you did nothing but say, I'm going to take all of my emails I've ever written that are still stored and give it to a vector store, use an embeddings algorithm, and then use ChatGPT to let me kind of answer questions. So if I want to say, oh, I want you to give me a timeline for when we first started using Hub to name products or whatever, and how'd that come about? Or what were the winning arguments against doing that versus whatever. Like, it's shocking how good the responses are when you give it access to that kind of rich data, right?

Sean Puri (12:42)

Somebody needs to create just like A$10 a month, a single website that's like, hey, make your chat GPT smarter. And it's a website where it's like, connect your Gmail, connect your slack, connect your everything. I would pay them happily 20 bucks a month to just set this up for me so that my chat to give my chat GPT. Like the extra pill that says you now have access to my data is this because. Because you're talking about, like, I have the API to the vector embeddings and like, well, I have the flux capacitor too, but I don't know what to do with it, right? Like, I need a button on a website with a stripe payment button that I could just connect the stuff. Is it not.

Sam Altman (13:16)

Is there. Is there a caveman version of this?

Dharmesh Shah (13:19)

I mean, there are tools out there and there are startups working on it, right? There's two pieces of good news. One is there are startups working on it. The challenge here is not that they're doing a bad job. The challenge actually comes down to if it were a startup and a startup came to you, it's like, oh, we just started last week, but we've got this thing. It really works. In fact, Dharmesh may be investor. How willing would you be to hand over literally your entire life and everything that's in your email over to this startup? So part of the challenge we have is that the access control that let's say you're using Gmail, which a lot of us use, when you provide the keys to your Gmail account to a third party, there is no degree of real granularity. You can say, oh, I wanted to read the metadata. That's like level one, Level two access. I wanted to read my full email. And level three is I wanted to be able to write and delete emails on my behalf. But if you wanted to like read like the actual body of the email, you can't say I only wanted to read messages that are from HubSpot.com, or I want to ignore all messages from my wife and my family or whatever in the thing. There's no way to control that. Right? So you sort of have to have trust.

Sam Altman (14:22)

Is there any product that you would trust right now or that you can recommend that guys like Sean and I should use as ChatGPT add ons or accelerators?

Dharmesh Shah (14:33)

No, not I. Not that I don't trust them, but it's like I wouldn't trust really anyone right now with that. And it's one of the reasons I sort of run it locally even though I know these things are out there. I predict what's going to happen is we're going to have any of the major players and you can see this happening already, right? We see this with you have the ability to create custom GPTs in OpenAI and do projects in Claude. You have Google Gems, which are essentially like a small baby version of this, right, that says, oh, you can upload 10 documents, 100 documents that it'll let you ask questions against. What it's really doing behind the scenes is creating a vector store. That's effectively what's happening. My expectation is all the major companies will actually have a variation of this, starting with Google should be the first one because they already have the data. There is absolutely zero reason why Dual Gemini does not let you have a Q and A with your own email account. That's just like insanely stupid, right? Like I'll just go ahead and say it. There's something not right with the world when they already have the data and it's like. And they have the algorithm. They have Gemini 2.5 Pro, which is an extra exceptionally good model, right? So they have all the pieces but have not yet delivered. But I hope is illustrating.

Sam Altman (15:43)

Then tell me and Sean, we're early adopters, but neither of us are technical. What can we do to. I want to get it on this baby.

Dharmesh Shah (15:51)

All right, so give me two weeks. Here's what we. So that's one thing I do trust and I trust myself. I'm an honest guy. I'll give you like this internal app that I'm building, let you point your Gmail to it. It'll go and it'll run for a day or two days or something like that. And then you will be amazed. You will be able to ask questions. And by the way, and the Thing I'm working on now is once you have this capability, step one is just being able to do Q and A. It's like, oh, I'm going to step to imagine fast forwarding. It has access to all of your history. So imagine you're able to say, you know what, I'm not doing this by the way, but if I were, it's like, I want to write a book about HubSpot and all the lessons learned and everything. It's all in my email. Do the best possible job you can writing a book. If you have questions along the way, ask me. But other than that, write the book. I think you'll be able to write the book. Wow.

Sean Puri (16:41)

What else are you doing with AI? So give me your day to day. For example, the CEO of Microsoft had this great thing where he goes, I think with AI, then I work with my coworkers. And that really shifted the way I worked because I used to brainstorm or have a meeting to talk about stuff with my coworkers, which was honestly always a little disappointing. I felt like I'm the one bringing the energy and the ideas and the questions and I'm hoping that they're going to. But dude, just sparring with AI first and then taking the kind of like distilled thoughts to my team of like, here's how we're going to execute has been way better. Like that little one sentence he said shifted the way I was doing it. How are you kind of using this stuff?

Dharmesh Shah (17:19)

Yeah, so a couple of things. So let's start at the high level and we'll drill in a little bit. So what we're used to with ChatGPT, this is sort of your kind of early evolution of most people's use is because it's called generative AI, you use it to generate things, right? Generate a blog post, generate an image, generate a video, generate audio, all those things. That's kind of the generation kind of aspect and that's part of what it's good at. Then you sort of get into the oh, but it can also kind of summarize and synthesize things. For me it's like, oh, take this large body of text, take this blog post, take this academic paper and summarize it in this way. Or like so a 7 year old would understand it kind of thing, right? So that's the kind of step number, step number two, step number three. And we're going to get into how this is now possible is you can do effectively, you can take action, have the LLM actually do Things for you. I'll just kind of put it broadly in the kind of automation bucket. Like I can automate things that I was doing manually before. And then the fourth thing is around orchestration. It's like, can I just have it manage a set of AI agents and we'll talk about agents in a little bit and just do it all for me, I just want to give it a super high order goal. It has access to an army of agents that are good at varying different things. I don't want to know about any of that. I just wanted to go do this thing for me, right? And that's sort of where we are on the slope of the curve. The first three things are possible today and work well today, right? So you can generate, as we know it can generate blog posts, it can write really well, it can generate great images now, including images with text. They can do great video now with higher fidelity, higher character cohesion, all these things. Sean, so the thing, the vision you had three years ago when I was on was around creating the next Disney, the next kind of media company. You have the tools now, my friend, to finally start to approach that, right? But then you should sort of move into. And this is what we were just talking about, this kind of synthesis and analysis thing. This is where deep research kinds of features come in. It's like, okay, well, I want you to take the entirety of the Internet or entirety of what Sean has written about copywriting, and I want you to write a book just for me that summarizes all of that in ways I enjoy, because I like analogies and I like jokes and I like this and I like that. Write a custom version of Sean Puri's book on copywriting. That kind of synthesis, I think would be super interesting. And then automation is now possible. So agent AI is one of those things. There's other tools out there that says, hey, I want to take this workflow or this thing that I do and I want you to just do it for me.

Sean Puri (19:34)

Give us a specific. What's a specific automation that you view that's useful, helpful, saves you time.

Dharmesh Shah (19:41)

I'll tell you a couple. One is around domain names, which is, okay, so I have an idea for a domain name and I'm going to type words in and these things exist. And I'll tell you, the manual flow that I used to go to is like, okay, first of all, I can brainstorm myself and come up with possible words and various types of words, whatever, here's the things. Then I'll say, okay, which domains Are available. Absolutely zero of them that are good, that will pop into my mind, are freely available to kind of just register that no one's registered before. Okay, fine. Then I'll say, okay, well, which ones are available for sale? What's the price tag? Is that a fair approximation of the value? Is it like below market? Above market? We don't know because there's no Zillow for domain names yet. So create that. So I have something that automates all of that and says, oh, so you have this particular idea for this concept for this business. Business, whatever it is. Here are names, here are the actual price points. Here's the ones that I think are below market value. Above market value. Tell me which ones you want to register.

Sam Altman (20:32)

That's in ChatGPT.

Dharmesh Shah (20:34)

No, it's in Agent AI, is where it lives right now. But now there's a connector between Agent AI and ChatGPT through this thing called MCP, which you'll hear about a bunch, if you haven't already. One thing I want to kind of get out there just so we keep connecting the dots, because I want everyone to have this framework in their head. So we talked about large language models. It can generate things. We talked about the context window. We talked about faking out the context window by saying, oh, we can do this vector database and bring in the right five documents, stuff them into the context window. Here's the other big breakthrough that's happened, I'll say recently, within the last year, year and a half, is what's called tool calling. And what tool calling is, is a really brilliant idea. And the tool calling says, okay, well, the LLM was trained on a certain number of things, but if we had this intern that came in, it would be like saying, okay, well, whatever, you know, you know, but we're not going to give you access to the Internet. Like, that would be stupid, right? We would give the intern access to the Internet. It's like, if I ask you something that you weren't trained on, go look it up, right? That would be like thing number one on the first day of work. And as it turns out, the LLM world, the internal, didn't have access to the Internet. All it had was whatever notes it happened to take during its PhD training and all things, right? And so what tool calling allows. And this is a weird approach to it, but this is because of the way LLMs work. So remember, the LLM, it's architected such that you give it the context window in, it spits things out. That's it. It doesn't have and you can't reprogram the architecture. Now all of a sudden we're going to give you access to tool calling. So here's the hack that they came up with. They said, okay, in the instructions that we give it in the context window, we're going to say you have access to these four tools. And it doesn't actually have access to the four tools. I want you to pretend like you have access to these four tools. The first tool is this thing called the Internet. And the way the Internet works is you type in the query and it will give you some things back. You have this other thing called a calculator and you can give it a mathematical expression and it gives you an answer back. And you have this other tool that lets you do this and you could have n number of tools. And so here's what happens in the context window happening behind the scenes. ChatGPT, which is the interface right now that is interacting with the ll. You're not talking with the LLM directly, right? It gets a prompt and it says, okay, by the way, LLM, I want you to pretend like you have access to these four tools. And anytime you need them, when you pass the note back to me, the results, the output, just tell me when you want to use one of those tools. All right, so we give it a query. It's like, okay, well, I want to look up like the historical stock valuation for HubSpot and when it changed as a result of. Is there any correlation to the weather, is it seasonal or whatever it is, right? In terms of market cap of HubSpot versus seasonal changes. All right, well, that's not something you would have access to, but here's what actually happens. This is so cool, right? So the LLM gets it and the LM's in the context window that we gave it, we gave it instructions. It's to pretend like you have these four tools, one of which is stock price lookup, let's say historical stock price lookup. It'll pass the output back to the application, not us, and say, and in the output it says, oh, please invoke that tool you told me I had access to and look up this result. I want you to search the Internet for X, what was the weather? I want you to do this for the stock price and then redo that. We, the ChatGPT application, fill the context window with whatever it is the LLM asked for and then pass it back in. So the LLM effectively has access to those tools, even though it never accessed the Internet. It never accessed the stock market, but it pretended like it had access to it. And we never see this. This is happening behind the scenes. Now. Here is the big massive unlock, right? Which is, well, everything can be a tool right now. You don't have to build this kind of vector store or whatever, because you would never build a vector store of all possible stock prices. From the dawn of timeout, I guess you could, but then it's outdated immediately. Now it's like, what if we just gave it 20 really powerful tools, including browser access to the Internet? Well, that's like a 10,000, 100,000 times increase in that intern's capability. That's where our brains should be headed now, which is exactly where the world is headed. That says, what tools can we give the LM access to that will amplify its ability and cause zero change to the actual architecture? Literally. It doesn't have to know anything about anything. I just want you to pretend that you have access to these tools. It doesn't need to know how to talk to those tools, doesn't need to know about API, doesn't need any of that stuff.

Sam Altman (24:53)

Cutting your sales cycle in half sounds pretty impossible, but that's exactly what Sandler training did with HubSpot. They use Breeze, HubSpot's AI tools to tailor every customer interaction without losing their personal touch. And the results were incredible. Click through rates jumped 25%, qualified leads quadrupled, and people spent three times longer on their landing pages. Go to HubSpot.com to see how Breeze can help your business grow. Do you think that, I mean, this is all mind blowing and you have an interesting perspective? Because, you know, I think three episodes ago that you're on, you created this thing called Wordle. Was it Wordle?

Dharmesh Shah (25:34)

Wordplay.

Sam Altman (25:35)

Wordplay, that does like 80 grand a month. It was just like a puzzle that you do with your son. It was amazing. But now you have new projects, you have agent AI, you have a few other things, but you still run a $30 billion company. Do you think that the majority of value creation, like, am I gonna. Is my stock portfolio going to go up because I own a basket of tech stocks, or is the best way to capitalize as an outsider, obviously you start a company, or is it investing in new startups that are using AI or AI for startups?

Dharmesh Shah (26:10)

Yeah, it's a big question. I'm neither an economist nor a stock analyst, but I will say this. The thing I'm most excited about with AI, and I actually said exactly this in a talk I gave, well, before GPT on the inbound stage. And I said, as AI is starting to kind of come up, it's not a you versus AI, that's not the mental model you should have in here. It's like, oh well, AI is going to take my job because it's me trying to do things that the AI is then eventually going to be able to do. The right mental frame of reference you should have, it's you to the power of AI. AI is an amplifier of your capability. It will unlock things and let you do things that you were never able to do before, as a result of which it's going to increase your value, not decrease it. Right? But in order for that to be true, you actually have to use it, you have to learn it, you have to experiment with it. And the only real way to get a feel for what it can and can't do is you have to do it. So I'll give you the very, very simple, everyone should do this, I do this personally, is that anytime you're going to sit down at a computer and do something, research whatever it is you're going to do, you should give ChatGPT or your AI tool of choice a shot at it. Try to describe and pretend like you have access to this intern that has a PhD in everything. It's like, okay, well maybe it doesn't know anything about me or whatever, fine, so then tell it a few things about you. But imagine you have access to this all knowing intern that has a PhD in everything. Give it a crack at solving the problem that you're about to sit down and spend some time on. And what you will invariably find, number one, is you'll be surprised by the number of times it actually comes up with a helpful response that you would never expected would be even remotely able to do. Like, how can it do that? It's because it has a PhD in everything, right? And now actually we'll talk about reasoning and whether models are actually doing that or not, if we have time. So that's my advice is every day, every day you should be in ChatGPT if you're a knowledge worker at all. It doesn't actually, you don't even have to be a knowledge worker. I don't care what your job is, right? You could be a sommelier at a restaurant and you should be using ChatGPT every day to make yourself better at whatever it is you do. And that might be the introduction of that orthogonal skill to bring it back to the which I never explained the word orthogonal. I'll do it in 30 seconds. So orthogonal means a line that's 90 degree intersection to another line. And the most common use is when we have an X and Y axis. Right. It's like, oh, the X axis and the Y axis are orthogonal to each other because they have 90 degrees separating them. The common usage, when you say, oh, that's an orthogonal concept, it means it's unrelated, it's completely different. That's like the Y and X axis are completely independent of each other. You can say, oh, you can be here on the X axis but here on the Y axis and they're not related to each other. So that's what I mean when I say orthogonal concepts or skills or ideas. Yeah.

Sean Puri (28:49)

Anyway, is there anything you disagree with that's kind of the consensus? Because a lot of things you're talking about, like, hey, AI is going to change everything. It's super smart agents are coming, they can do some stuff now, more stuff later. These are all probably right, but they're also consensus. I'm just curious, like, is there anything you disagree with that you hear out there that drives you nuts, where you're just like, people keep saying this. I think that's either wrong, it's overrated, it's the wrong timeline, it's the wrong frame, it's whatever. Is there anything that you disagree with that you've heard out there?

Dharmesh Shah (29:19)

I've heard variations. Two variations I disagree with. One that I've I think spent some amount of time hopefully kind of talking folks out of which is it's just autocorrect. It's not really thinking. And that's a matter of what do you think thinking is right? It's like, okay, well if it produces the right output to which we think would require thought. So I think that is flawed reasoning to say, oh well. And this often comes from the smartest people, the most experts in their field because, oh, it's really like a stochastic parrot. You'll hear this phrase which is it's like a probability driven pattern matching based. It just so happens that's been trained on the Internet. But it's not really like human intelligence. And I agree with that phrasing, which is it's not like human intelligence. But that does not mean that all it's doing is sort of mimicking stochastically all the things I read before because in order to do what it does, it is a form of creativity different from what we normally experience. That's kind of thing number one that I disagree with. Thing number two is people are Thinking I both disagree with the oh, the scaling laws are going to continue forever, indefinitely. That the more and more compute we throw at, the more knobs we put on the machine, the smarter, smarter it's going to get. I think there's going to be a limit to that at some point, as if nothing goes on forever, it's going to asymptotically move towards. We're going to have to come up with new algorithms. GPT can't be the dual end of all things, right? There'll be a new way discovered. So I think that's going to happen. But I think the smarter and I did not say this, other people have said it, the best way to kind of think about AI right now is as you use it is to kind of truly find a frontier of what it's incapable of. It's like, okay, it can sort of do this thing, but not very well. If that's the way you describe its response, you are exactly where you need to be. Which is if it can sort of do it right now, sort of, if you have to squint a little bit, it's like, ah, it's kind of something. But wait six months or a year, right? That's the beauty of an exponential curve. It gets so much better so fast that if it can sort of do it now, it will be able to do it and then it'll be able to do it really well. That's the inevitable sequence of events that's going to happen.

Sean Puri (31:24)

Have you heard this about startups? There's like a kind of the smart money in startups believes that the right startup to build is basically the thing that AI kind of can't do right now. That's the company to start today because you just have to stay alive long enough, give it the, the 12 to 18 month Runway that it needs for the thing to go from didn't really work very well to like, oh my God, this is amazing. But you've built your brand, your company, your mission, you've your customer base, you've been building that all along the way and you're basically just betting you're going to be able to surf the improvement of the.

Dharmesh Shah (31:55)

By the way, that's, that's how I.

Sam Altman (31:57)

Feel about my company. My company is not related to this but at all. But in terms of like our operations were like things are very manual and I'm like, oh my God. Once I'm able to finally implement AI, when it can work for this purpose, my profit margins would go through the roof. I mean that's how, that's how I feel about it, but which isn't entirely related to that, Sean, but a little bit.

Dharmesh Shah (32:16)

One, one, one thing I'll plant out there since this is my first million wheat. We like talking about ideas at a macro level. Here's the entirely new pool of ideas that I think are now available on a trend that I think is inevitable, which is as agents get better and better. Right now, most of us, when we use AI, use ChatGPT, we use them as tools, which is great, perfect, fine. Over time, you need to shift your thinking and think of them as teammates. Think of them as that intern that just got hired. Right. And as a result of that, so let's assume for a second, let's stipulate that I'm right. All we don't know is how long is it going to take for me to be right, is that we're going to have effectively digital teammates that are part of all of our teams. Every company is going to someday have a hybrid team consisting of carbon based life forms and these kind of digital AI agents. Okay? So if you accept that the way that's going to happen is not going to be like all of a sudden we one day wake up and every organization now starts kind of mixing them, what's going to happen is it's going to slowly introduce this waste, like, oh, I have this one task. Whatever that agent is better at, it's reliable enough for the thing and the risk is low enough. I'm going to have it do that. But we already see elements of that. But here's what's going to happen as a result of that kind of gradual kind of infusion and adoption of that technology. The way to win and the opportunities that get created is like, how do I help the world accomplish this end state that I know is going to come. So here I'll give you some examples. If we were to hire, if you, Sam, were to hire a new employee tomorrow, here's what you would do. You would say, oh well, I'm going to onboard that employee, spend a couple of days, I'm going to tell them about the business. Whoever's managing that employee, let's say with a direct report of yours, maybe you'll have a weekly one on one or every other week or whatever. That one on one will consist of looking at the work they did. Whatever it's like, oh, over here, you did this or whatever. And it could be copy editing, it could be anything. Whatever the role happens to be, you're going to give them feedback Right. That's what you would do for a human worker. All of those things have a direct, literally a direct analog in the agent world. And what we're doing right now is we're hiring these agents and expecting them to do magic. Just like if we hired an exceptionally Smart, has a PhD in everything employee and expected them to do magic with no training, no onboarding, no feedback, no one on one, no nothing. Well, your results are not going to vary. They're going to be crap because you did not make the investment in getting that agent. Now, the big unlock here. So whether you're an HR person or whatever, it's like figure out, well, what does employee training look like for digital workers? What do performance reviews look like for digital workers? How do we do recruiting for digital workers? How do we, like, what are all the mechanisms that need to exist? What is a manager of the future? What are the new roles that will be created as a result of having these hybrid teams? It's like, okay, well now maybe we're going to need someone that's like the agentic manager human that knows all the agents that are on their team or whatever and has kind of built the skillset how to do recruiting for their team, how to do performance reviews, how to do all of that. But for agents or hybrid teams versus just purely human ones, that's just a whole other. And we're going to need the software, we're going to need the onboarding, we're going to need training, we're going to need books written, we're going to need all of it to kind of adopt. And it's going to take years, right? It's not happening overnight.

Sam Altman (35:28)

Two years ago I asked you, is it going to be as bad or I think you said, I asked, is it going to be horrible or is this going to be amazing? And you said, I saw this with the Internet. Nothing is as extreme as the most extreme predictions. I listened to you and I trusted you then. I actually think, knowing what I know now, I'm actually more fearful than I was a couple years ago where I'm like, oh, this is actually going to put a lot of people out of work and it's maybe not good or bad, but things are going to change drastically more than I thought. And Mike, so I don't remember how I phrased the question, but is this going to change the future more than you thought two years ago or less than you thought two years ago? Has your opinion on that changed?

Dharmesh Shah (36:12)

I still think they're going to be unrecognizable My kind of macro level sense, and this is maybe just my inherent optimism about things, is that it's going to be kind of a net positive for humanity. And this is the other thing that lots of people would disagree with me on that's like, oh, well, is this an existential crisis to the species? And I've not said this before, but I'm going to see how it sounds as the words leave my mouth. I'm probably going to regret it. But in a way we are actually. And Sam, Shawn, you said this earlier, we're sort of producing a new species, right? So that's like saying, okay, well homeo sapiens as they exist, absent AI is likely not going to exist. So the way we know the species as it exists today, with where we have a single brain in natural form, four appendages or whatever, maybe that's going to be different. But I think of that as an extension of humanity, not the obliteration of humanity. Right. That's human 2.0 or n.0 of the way we kind of think of the species right now. So I think things are still moving very, very fast. And this is why I think humans have issues with exponential curves. We're just not used to them. When something is kind of doubling or every n months, it's hard to wrap our brains around how fast this stuff, you know, can move. Things that we thought were like the things we have today, Sam, if we had just described them to someone a year and a half ago, there's like, ah, well, chatgpt is cool or whatever, but it's never going to be able to do that. And now we're like, those are like par for the course, right? Like we can do like things that were literally like, oh, there's no way, no way. It's like, yeah, it's good at like text and stuff like that. But that's because it's been trained on text. Now I can do images. Well, I can do images, but like video is like 30 frames a second. That's like generating 30 images per frame per second of videos. Like all of that. It's like, yeah, but diffusion models, the way they work is because you're not going to get, you get a different image every time. So how are you going to create a video? Because it requires the same character, the same setting in subsequent frames. That's not how the thing is archived. That's not how image models work. And we solved all of those things. Right now we have character cohesion, setting cohesion and video generational. Anyway, so my answer is it's exactly. Not exactly, but it's close to like, yep, this is what exponential advancement looks like. I'm still of the belief that we're going to have more net positive. That is not to say that in the interim there's not going to be pain. And there's two things I'll put out there as cautionary words. One is, in the interim, anyone that tells you that there's not going to be job dislocation, there's not going to be roles that get completely obliterated is lying to you, that is going to happen. It's already happening, right? It's that there is no world in which that does not occur. That's kind of thing number one. Thing number two, and we didn't talk about this, but we should have, is that because of the architecture of how LLMs currently work, maybe they'll figure out a way to do that. They produce hallucinations. That's just a fancy way of saying it makes things up and that's sort of okay, but not okay because it doesn't know it's making it up because of the way the architecture works. It's like the Internet that thinks it's been exposed to all there is to know in the world. It's like, I know all the things. You ask me a question, I know I know all the things. So I'm going to tell you the thing that I know. It was like, well, yeah, but you didn't know this. And what you said is actually factually, like provably, demonstrably wrong. And it has absolutely zero lack of confidence in its output. Which is fine for some things. If you're writing a short fiction story or something like that. It's not great at all for other things like healthcare related, where you need kind of predictable, accurate responses. So I think we need to be aware of the limitations around it when we're doing research and things like that. And the problem is when we have relatively, I'll say naive, and I don't mean this in a disparaging way. Folks that are naive to a subject area asking ChatGPT for things where it can't judge the response, right. We're sort of taking it on faith that it's ChatGPT and their mesh, that it's got a PhD and everything. So of course it's going to be right. Well, no, it's often not right. And it's kind of up to us to figure out what our kind of risk tolerance is. It's like when is it okay for it to be wrong? How would I test it for my domain for my particular use cases? Yeah, so.

Sam Altman (40:20)

So you guys know this, but I have a company called Hampton. Join Hampton.com, it's a vetted community for founders and CEOs. Well, we have this member named Levon. And Levon saw a bunch of members talking about the same problem within Hampton, which is that they spent hours manually moving data into a PDF. It's tedious, it's annoying, and it's a waste of time. And so Levaan, like any great entrepreneur, he built a solution. And that solution is called Moku. Moku uses AI to automatically transfer data from any document into a PDF. And so if you need to turn a supplier invoice into a customer quote or move info from an application into a contract, you just put a file into Moku and it auto fills the output PDF in seconds. And a little backstory for all the tech nerds out there. Slavon built the entire web app without using a line of code. He used something called Bubble IO. They've added AI tools that can generate an entire app from one prompt. It's pretty amazing. And it means you can build tools like Moku very fast without knowing how to code. And so if you're tired of copying and pasting between documents or paying people to do that for you, check out Molku AI. M O L K U DOT AI all right, back to the pod.

Sean Puri (41:26)

What do you think about this situation where Zuck is throw in the bag at every researcher, 100 million dollar signing bonuses, even more than that in comp, and he's poaching basically his own dream team. He's like, okay, you're not gonna. I can't acquire the company. Well, why don't I go get all the players? If you can keep the team, I'll keep the players. And he's going after them with these crazy nine figure offers.

Sam Altman (41:49)

100 million signing bonus and 300 million over four years, I think is what I saw. Is that true?

Sean Puri (41:54)

I think that was like the higher. Yeah, so the higher end. And some people have said there's even like billion dollar offers to certain people that are out there. This is like job offers. So Dharmesh, like, were you shocked by this? Because I mean, my reaction to this was that's bullshit. First time I heard it, then I was like, wait, the source is Sam Altman? Why would he say that? And then I was like, okay, that's insane. And then an hour later I was like, wait, that's Actually genius, because for a total of 3 billion or something, he can acquire the equivalent of one of these labs that's valued at 30, 40, 50, or $200 billion. What a power play. I know, obviously you're investor in OpenAI, so, you know, maybe you don't like this, maybe you have a different bias here, but I'm just from one kind of like leader of a tech company to another, like, what's your view of this move? I think it's one of the crazier moves.

Dharmesh Shah (42:40)

If I had to use one word, I would say diabolical. Not stupid, not silly, but diabolical. And here's why, right? This is the, like in the grand scheme of things. So this is not just a, oh, can we use this technology and build a better product that will then drive X billion dollars of revenue through whatever business model we happen to have. There's a meta thing at play here that says whoever gets to this first will be able to produce companies with billions of dollars of revenue or whatever. Right? Because that's like kind of finding the secrets of the universe, the mystery of life kind of thing. It's like, okay, well, whoever wins that and gets there first will then be able to use the technology internally for a little while and be able to just kind of run the table for as long as they want. So it's got incalculable value. Right? The upside is just so high that no amount of, like, if you can increase your probability even by a marginal amount, if you had the cash, why wouldn't you do it? Right?

Sean Puri (43:31)

So do you think a. Do you think it'll work? Do you think this tactic will work for him? Do you think he will be able to build a super team? Is he just going to get a bunch of engineers who now have yachts and don't work? Like, what's going to happen when you give somebody $100 million offers? You put together this smash together, this team of. I think he's got a hit list of 50 targets. And I think, like, you know, something like 19 or 20 of them have come on board already. What's your prediction of how this plays out?

Dharmesh Shah (43:56)

It feels a little bit like a Hail Mary pass, right? That's okay. They're going to take this. It's like, okay, well, there's not a whole lot of things we can do. You know, the chips are down. I'm going to mix metaphors now, too.

Sam Altman (44:06)

But that works.

Dharmesh Shah (44:07)

Sometimes it works. Sometimes. That's exactly why people do it. People get it. What other option do we have, right. Like everything else hasn't worked yet, so let's try this thing. But I think the challenge, I still think it's a diabolically smart move. I'm not going to use the word ethics or anything like that. But here's the challenge though, right? If we were having this conversation, we'll call it two years ago, give or take, OpenAI was so far ahead in terms of the underlying algorithm, and this is even before ChatGPT hit the kind of revenue curve that it's hit just raw. The GPT algorithm was just so good and they were so far ahead, it was actually inconceivable for folks, including me, that others would catch up. It's like, okay, well they'll make progress, they'll get closer, but then OpenAI is obviously going to still keep working and they're going to be far ahead for a long, long time. That's proven not to be true. Right? We've seen open source models come out, we've seen other commercial models come out. There's anthropic and they have by most measures comparable large language models. Right? Within one standard deviation, they're pretty good. And sometimes they're better at some things, worse at others, but it's not this single horse race anymore. So the thing that I'm a little bit dubious of is that even if you did this, you put all these people together like it didn't really work for OpenAI in the true sense of the word, right? Like they weren't able to create this kind of magical thing that it's like, okay, maybe they end up doing it somewhere else, but I think there's more smart people out there. The technology is kind of deep seek prove that you could actually. And they actually have an actual innovation in terms of reasoning models and things like that versus kind of the early generation large language models. So jury's still out.

Sam Altman (45:40)

How much better is a $300 million over. So $100 million a year engineer over like a $20 million engineer? Is it like, I followed some of these guys on Twitter and it was, they're fantastic follows and do you think that their IQ is just so much better or is it because they've had experience? Is it really because they just saw how OpenAI works and they want that experience? Are they like, is this like espionage? What is, how good could a hundred million or three hundred million dollars a year engineer be?

Dharmesh Shah (46:11)

Well, that's the thing though. This is software, right? So this is a world of like 95% margins. So let's say I think part of the value is, yes, they're super smart, but even human IQ asymptotically moves towards a certain ceiling. Right. You take the smartest people in the world, however you want to measure iq, and so that doesn't explain away the value. That's not that. It's not that. They've seen the inside of OpenAI and they have some trade secrets in their head that they can then kind of carry over. It's like, oh, here's how we did it over there, here's how we ran evals, and here's how we did the engineering process. They'll have some of that because we always carry some amount of experience in our heads. I think the larger thing, I think the primary vector of value is they sort of have demonstrated the ability to kind of see around corners and see into the future. Right. They believed in this thing that almost no one believed in at the time. They sort of saw where it was headed and they were working at it, chipping away at it, whatever. And that's much rarer than you would think for really smart people to do this stupidly foolish, seemingly stupid, foolish thing. It's like, you're going to do what now? Right? And we're still asking ourselves a variation of that question that we would have asked three years ago, except now we have ChatGPT and we have the things that. And we're still like, well, you say that we're going to have like these kind of digital teammates and they're going to be able to do all these things and it can't even do this simple thing. Right? Right. Like, we sort of keep elevating our expectations and what we believe is or is not possible. They sort of know what's possible, and they almost think of what many of us would consider impossible as actually being inevitable.

Sam Altman (47:37)

Have you guys. Has HubSpot, have you made any of these offers?

Dharmesh Shah (47:40)

I don't think so. But that's not the game we're in. Right. So we're not in that league. We're not trying to build a frontier model. We're not trying to invent AGI. We're at the application layer of the stack. So we want to benefit from it. Right. We didn't. In any layer of my entrepreneurial career, I have not been the guy in the center of the universe sort of company.

Sam Altman (47:59)

But you're not like, oh, man, I met this person, like, we need to offer like an NBA contract in order to secure this. This guy.

Dharmesh Shah (48:06)

No. And there's a reason for this, right? It's like for the kinds of problems we're solving, what's the. There's a sports term about the best alternative to the player or something like that, that a replacement cost more, wins.

Sean Puri (48:15)

Above replacement is the metric they use in sports.

Dharmesh Shah (48:18)

So yeah, it's just not, it's not worth it. Given our business model and given what we do. I have one last thing on the kind of AI front. This is one of the things answering your question, Sean, in terms of things I disagree with folks on, is that there's a group of people, very smart that will say, oh well, AI is going to lead to a reduction in creativity, broadly speaking. Right? Because you're just going to have AI do the thing. Why do you need to learn to do the thing? And I have a 14 year old, right? So it's like, okay, well if he just uses AI to write his essays and do his homework or whatever, it's going to reduce this creativity. And I understand that particular kind of line of reasoning that says, yeah, if you just have it do the thing, you're not going to. But I think the part those folks are missing is that creativity is kind of in the literal sense, the word is like, okay, I have this kind of thing idea in my head and I'm going to express it in some creative form, be it music, be it art, be it whatever it happens to be. And the problem right now is that whatever creative ideas we have in our head are limited in terms of how we can manifest them based on our emerging skillset. So Sean can have a song in his head right now that like, he may be composing things in his head, but until he learns the mechanics of how to actually play an instrument, whatever the instrument happens to be, there's no real way to manifest that. Right. We can't tap into his brain and do that. So in my mind, AI actually increases creativity because it will increase the percentage of ideas that people have in their heads that they will then be able to manifest regardless of what their skills are or not. And I love that. So my son, he's a big Japanese culture fan, big manga fan and Japanese comic books and anime. And so he's an aspiring author someday and what he can do now. And he's been able to do this for years, which is so he's always had again, he likes fantasy fiction as well. So he's had these ideas for writing things, but he lacked the writing skills. He doesn't know about character development, doesn't know about any of these things. So what he uses ChatGPT for is he's got this like 2000 word prompt that describes his fictional world. Here are the characters, here's a power structure, here are the powers people have, here's what you can and can't do. And then the way he tests the world is he turns it into a role playing game. It's like, okay, I'm going to jump into the world now. You chatgpt. I'm going to do this. Tell me what happens. Oh, this happened. Okay, now I'm going to do this. Okay, well now you've got this power. And so it will sort of kind of pressure test kind of his world. And so that's an expression of his creativity because the world was sitting in his head. But now he can actually share that with friends, maybe turn that into a book someday because it's going to take the ideas that he has and hopefully in the meantime he will kind of develop some of those foundational skills. But he doesn't have to wait until like 12 years of writing education before he can take this idea. As a child he has lots of creativity, but as a practitioner, most of those things that he would love to be able to manifest in the world. He has nothing close to the skills required, whether it's drawing or writing or anything. So I think that's what AI can help us kind of elevate once again. But we have to use it responsibly, but it should be able to elevate our skills.

Sean Puri (51:18)

I want to show you guys a example of this real quick. So I had this idea not long ago, a couple weeks ago of creating a game using only AI. So I don't know if you guys ever played the Monkey island games from like when I was a kid I played Monkey Island. It was a incredible game. It's got basically this guy wants to be a pirate. It's like this very funny but like 8 bit art style game. And so I created a version of that called Escape from Silicon Valley. I didn't create the whole game but I create like the art. But like check this out. So, so I go into, I go into AI and I basically start creating the game art. And so it's like the story is basically like deep in San Francisco. The year is 2048. The Rock is starting his third term in office. You know, Nancy Pelosi passes away the richest woman on earth. And then you know, Elon is promising that self driving cars are coming really really soon for, for real this time. And here you are, you're this character and you're in the OpenAI office. And basically the idea is Charlie.

Sam Altman (52:17)

Look at that.

Sean Puri (52:18)

What's that?

Sam Altman (52:19)

Look at the Charlie bar.

Sean Puri (52:21)

Yeah, yeah, exactly. I was putting in some references to like, you know, stuff that I thought was it would be cool.

Sam Altman (52:26)

That is so cool. What did you use to make those images?

Sean Puri (52:29)

So that right there was just chatgpt and Midjourney Mix. I tried using scenario and a couple other game specific tools. Check this out. So I created all these characters. So it's like I create Zuck and Palmer Lucky and Chamath and Elizabeth Holmes in jail.

Sam Altman (52:44)

Awesome.

Sean Puri (52:44)

And I had it basically write the scenes for the levels with me, write the dialogue with me, create the character art.

Dharmesh Shah (52:50)

Dude, that looks sick.

Sam Altman (52:51)

Why didn't you do that?

Sean Puri (52:53)

Well, because I did the fun part in the first two weeks where I was like, oh, good concept. The levels, the character art, the music, seeing what AI could do. But then to actually make the game, the AI can't do that. And so I was like, oh, now I need to like. I mean, people who build games and years building it. It's like, oh, this is like minimum 6 to 12 months doing this, like, very, very arbitrary project. But I still love the idea. And I'm kind of like packaged up the whole, the whole idea.

Sam Altman (53:20)

Dharmesh, last question. Just really quick, like you, where do you hang out on the Internet that we and the listener can hang out to stay on top of some of this stuff? Like, are there like, who's a reputable handful of people on Twitter to follow or reputable websites or places to hang out at?

Dharmesh Shah (53:39)

That's interesting. So I spend most of my time on YouTube, as it turns out, and I sort of given to the vibe, so to speak, and let the algorithm sort of figure out what things I might enjoy. Gets it right sometimes, gets it wrong sometimes. So it's a mix of things. But the person that I think if you want to kind of get deeper into like understanding AI, there's a guy named Andrej Karpathy. I don't know if you've come across him. Just search for Karpathy, dude, you don't.

Sam Altman (54:10)

Want to know how I know. Like, I get so many ads that says, like, Andre Karpathy said this is the best product, or Andrej Karpathy showed me how to do this. Now I'm going to show you. I don't even know who Andre is other than ads run his name to promote him.

Dharmesh Shah (54:24)

Yeah, I mean, he's one of the true OGs in AI, but he has that. His orthogonal skill or one of them. I think he's got like nine. He's probably like a nine tool player of some sort, but he's able to really simplify complicated things without making you feel stupid. Right. So he's not talking down to you. He's like, okay, like, here's how we're going to do this. We're going to kind of build it brick by brick and you're going to understand at the end of this hour and a half how X works. Right. And it's. And he's amazing.

Sam Altman (54:51)

So that would be one. So him. Any other YouTubers or Twitter people or.

Dharmesh Shah (54:55)

Blogs on the business side? Actually, Aaron Levy from Box is actually very, very thoughtful on the. If you're in software in business and the AI implications there, I think he's really good. Hiten Shah, who you both know now at Dropbox through the acquisition, has been on fire lately on LinkedIn. So he's one. I would go back, especially the last three, four months, and read all the stuff he's written. I think he's on point. Yeah.

Sean Puri (55:19)

So those are awesome. Dharmash, thanks for coming on. Thanks for teaching us. You're one of my favorite teachers and entertainer, so thank you for, for coming on, man.

Dharmesh Shah (55:28)

My pleasure. It was good to see you guys. It was fun.

Sam Altman (55:30)

Likewise. Thank you. That's it. That's the pod.

Dharmesh Shah (55:34)

I feel like I can rule the world I know I could be what I want to I put my all.

Sam Altman (55:39)

In it like no days off on.

Dharmesh Shah (55:41)

A road less travel never looking back.

Sam Altman (55:44)

All right, my friends, I have a new podcast for you guys to check out. It's called Content is Profit and it's hosted by Luis and Fonzie Cameo. After years of building content teams and frameworks for companies like Red Bull and Orange Theory Fitness, Luis and Fonzie are on a mission to bridge the gap between content and revenue. In each episode, you're going to hear from top entrepreneurs and creators and you're going to hear them share their secrets and strategies to turn their content into. Into profit. So you can check out Content is Profit wherever you get your podcasts.