Loading summary
Transcript75 lines
- [00:01]
A
Support for the show comes from Odoo. Running a business takes everything you've got, and a lot of the tools out there that are supposed to make your life easier just aren't great at talking to each other. And that means you end up having to toggle between a dozen different apps and services just to keep the lights on. Enough of that. Now there is Odoo, the all in one fully integrated platform that might actually help you get it all done. Thousands of businesses have made the switch, so why not you try Odoo for free@odoo.com that's O-.
- [00:39]
B
Support for the show comes from Plod. If you're an executive, small business owner, project manager, journalist, or anyone responsible for important decisions, details matter, conversations move fast, and it's easy for key contacts, follow ups and action items to slip through the cracks once the meeting ends. Plod is an AI powered note taking system built around dedicated recording hardware. It captures conversations, transcribes them and turns them into searchable transcript summaries and action items so you can focus on the discussion instead of worrying about capturing every detail yourself. Visit Plaid AI Markets to learn more and use the markets code to get up to 15% off.
- [01:20]
C
Support for this show comes from Odoo. Running a business is hard enough, so why make it harder with a dozen different apps that don't talk to each other? Introducing Odoo. It's the only, only business software you'll ever need. It's an all in one fully integrated platform that makes your work easier. CRM, accounting, inventory, E commerce, and more. And the best part? Odoo replaces multiple expensive platforms for a fraction of the cost. That's why over thousands of businesses have made the switch. So why not you try Odoo for free@odoo.com that's o d o o.com.
- [02:12]
A
Welcome to Prof. G Markets. We've spent a lot of time talking about AI lately, from the Trump administration's export restrictions on anthropics models to the ongoing questions surrounding the economics of companies like OpenAI and Anthropic. Taken together, these stories point to two fundamental questions. One, is the AI boom financially sustainable? And two, are we moving too quickly with a technology that we don't fully understand? Few people have been asking those questions longer than our next guest. Long before concerns about AI safety, regulation and business models entered the mainstream, he was warning about the technology's limitations and challenging some of the industry's most ambitious claims. He he has testified before the Senate on the risks posed by AI. He's founded a machine learning company that was acquired by Uber and is now one of the field's most prominent skeptical voices. So here's our conversation with Gary Marcus, AI skeptic, author and professor at NYU Stern. Gary, thank you so much for joining me on the show today. You are one of the original critics of AI, and that's quite interesting because you're also. You work in AI, you started a machine learning company, which I think you could say is an AI company. You've done a lot of AI research. You are sort of part of the AI world, but you have issues with it. Let's just start broad. What are your concerns?
- [03:45]
C
Well, my concerns are we're all in on a particular technology that I think is inelegant, harmful, not where we should wind up and being abused by the people using it. So I want AI to succeed, but I think we wound up down this really dangerous path. So you think about the Star Trek computer. You ask it a question, it gives you an answer that you can count on. Presumably it's not done to sort of wreck society. It's done to help people. Well, what we actually have is everybody running around with LLMs, which are inherently unreliable. They're unpredictable, they can't be aligned to human values. And they're being run by companies that don't seem to really give a shit about the consequences for what they're building for society. It's like a nightmare for those of us who have worked in AI to suddenly see what we're building, be used in so many bad ways and with people really not caring. You know, we should want a more reliable technology that we can really count on that is compatible with humanity. You know, five years ago, that didn't seem out of the question. Five years ago, the field was healthy. It was considering lots of different things, wasn't driven so much by money, but by intellectual curiosity. How do you make a machine that's intelligent? And everything changed when people started to realize that there might be money to be made. Still not clear that there actually is money to be made, by the way. Right. And I'm sure we'll get into that because we have very similar views about that. But the thought, the scent of money, possibly misguided scent of money, really changed how the field grew. And it's also, you know, a technical thing. Transformers are interesting and people got into them. But fundamentally, I think it's the scent of money really changed how people built AI, how they thought about it, what they wanted to do with it, who was running it. I think A lot of grifters came in that don't even necessarily have technical understanding of the questions and do a lot of lying and hyping about what their things might actually do. And it's just really been unpleasant for the last several years being honest about it. And it's not because I don't want AI to succeed. I still think that there's a chance that AI could help a lot in medicine, that it could help with all kinds of technologies. Like I would still like to see AI succeed, but not on the path that we are right now. This is just not a good path.
- [06:07]
A
You recently wrote, you said, quote, generative AI has been inherently unreliable from the start. None of the problems that I warned about over the last half decade have been properly solved. There's the financial question, which we will get into, but then there's also the question of the technology itself. What do you see as the problem? What makes gen AI inherently unreliable? What is the path that you are worried about this technology going down?
- [06:37]
C
The technical problem is that large language models, fewer large language models are basically next token predictors. That's what they do, that is literally how they are built, is to predict in a sequence of words or other kinds of tokens what might come next. And that's an interesting thing to do. It's part of what humans do is we do some prediction, but it's not all of what cognition is. Right? Cognition. Intelligence is about cognition, about understanding things and so forth has many different components to it and they're just not really built into LLMs. And so LLMs basically fake everything else. And we can talk about some complications people are building in harnesses and we can go there. But let's just Talk about Pure LLMs. What they do is predict the next token. And if you train them on the entire Internet, which is what people in fact do, they can make a pretty good approximation of human beings and how they talk and so forth. But that approximation is very superficial, it's very data dependent. And when you push them outside of the regime in which they've been trained, they will do really stupid things. So like a couple years ago there were all these examples of so called river crossing problems, like you have a man and a goat and a woman and they have to go across the river. And these systems would say the most absurd things in response to those problems. It got so embarrassing that anthropic built in river crossing problems into their system prompts to try to keep the systems from making these embarrassing errors. And what the embarrassing errors revealed is the systems are not really reasoning about things like a man or a river or a boat or what it means to go across the other side. They're just trying to kind of glom the words together that they have seen. I mean, the technical details are a little bit complicated, but to a first approximation, what they are doing is just stringing these words together. There are other ways to think about ability, intelligence. So you might start, for example, with a database. Who did what to whom, when and where. If you actually did that, if you started with that, you would not have all these crazy hallucinations. And so here we are, you know, in 2026. I started writing about LLMs in 2019, and I said, they don't have stable models of the world. You can't trust on them. And everybody said, gary, Gary, Gary, we're just going to add more data. All these problems are going to go away. Hallucinations are going to go away. Mustafa Suleiman, who's the CEO of AI or whatever his title is at Microsoft, said, you know, they're going to go away in a few months. This was in 2023, I think. I offered him a bet and he kind of walked back what it was. Reid Hoffman said he would bet any amount of money that hallucinations would go away in a few months. This is 2023. I said, I'm over here. How about $100,000? He never got back to me. But here we are in 2026, and hallucinations have not gone away. It's because the core of next token prediction does not allow you to address that problem. So you have to add something else. And something else rarely works all that well. It sort of works a little bit. I just saw a study yesterday showing there's a new benchmarking. I think it's called halu hard hallucination hard. And all the systems are still making errors on this, and none of this has gone away. It's hard for people who are not trained in cognitive science and artificial intelligence to understand when they play with these systems that they don't think like human beings, that they're really operating over different principles because they are built to mimic human beings. And human beings are not built to distinguish AI systems that work differently from themselves from actual humans. We have a lot of evolutionary machinery to find fast things that are moving that might be snakes or bugs or lions. We have nothing built into our brain to really help us think about the nature of intelligence. And so people are very easily fooled. We've actually known that for a long Time. We've known it for 60 years. Eliza was the first example of an AI system that could fool an average person into thinking it was much more intelligent than it was. Eliza behaved as a psychiatrist and he just did simple keyword matching. So you say, you know, relationship and it asks you to tell you more about that relationship or whatever just by matching keywords, not understanding anything. So Weizenbaum wrote about this in the 60s, how we are vulnerable to over attributing is the technical term intelligence to machines. And that was a curiosity I guess when he wrote about that in the 60s. But now that is the whole world, right? The entire economy. This is where our shared interest is. I suppose the entire economy is based right now is hinge hinging on over attribution of intelligence to these machines, right? You have people betting trillions of dollars that these machines are intelligent in ways that they aren't actually. Because those people placing the trillion dollar bets don't have enough cognitive science background to know the right test in order to evaluate intelligence. And then we have like government policies built around these things or considered around these things. The entire world is over attributing intelligence to LLMs. It's not that do anything like they're great for autocomplete for the purposes of computer coding and they're great for certain kinds of brainstorming and so forth. But their intelligence is still limited. And we probably need a completely different approach. I like to have a metaphor of climbing mountains, right? And you could get to the peak of one mountain and think, well I must be close to the top. But actually you might not be right. If it's a mountain range and there's a whole bunch of different peaks, you might be at the peak of 1. In order to get to the tallest peak, you might have to actually go back down the valley. And that's what we need to do. We actually need to give up some of the progress that we've made in order to come up with new ideas. But everybody's obsessed with one idea. They're obsessed with the large language model that actually has these problems of and we didn't even get into but bias and unreliability, et cetera. But people are so addicted to the one thing that they're all in on that. And we'll get to the economics soon, I suppose. But part of the economic problem hinges from that, that everybody is using the same solution. If you had healthy ecosystem, you might have 100 different companies trying 100 different approaches and you could say let the best one win. But we have basically 100 companies, maybe not 100, but a dozen companies, doing exactly the same thing. And if it's not the right thing, that's a problem. And even if it is the right thing, it's a problem. I'm pretty sure it's not, but it's still a problem. If everybody's doing the same thing, that means making profits is really hard. So we'll talk about the economics and why nobody is making profit. But the underlying reason nobody's making profit is they're all doing the same thing. If we all have the same toothpaste, nobody's going to pay that much for it. You can't charge $100 for a tube of toothpaste if. If you have, you know, nine competitors building basically the same thing for less.
- [13:40]
A
Basically, we're all using one of two models. Essentially. It's. You're probably using OpenAI's model, you're probably using Anthropic's model. And then a lot of these companies are building wrappers and building all of these gadgets and gizmos on top of those models. But to your point, you're basically just. Just putting rappers on top of the same fundamental thing.
- [14:00]
C
Well, and those two are actually basically the same. And they're basically the same as what?
- [14:04]
A
It's the same cognitive architecture.
- [14:05]
C
Yeah. They're small differences, but they don't persist. Right. I mean, that's another thing that we have seen over the last few years. I wrote this tweet, I think, in 2024, describing what I said is going to be a new regime where basically LLMs are going to run out of headroom, Everybody's going to wind up building essentially the same thing. There's going to be no moat between them, and that's gonna lead to price wars, and it's gonna lead to, you know, no huge difference between them. That's what we've seen is a lead that goes back and forth. Right. Somebody's ahead for a week and they pay, like, you know, $100 billion or whatever. The numbers aren't public, but, you know. Or the $10 billion, I guess, would be more plausible in order to get that lead that lasts like three weeks. Like, that's insane.
- [14:48]
A
Exactly. And then depending on which engineer you talk to, some would say Codex is better, and then some say Claude is better. Claude is coming out with. With slop. And then people say that OpenAI is coming out with slop, et cetera. I think one of the main points that you put forward, which I think is true by the way, is that these AI models are actually dumber than many of us would like to believe, especially those of us who are financially incentivized to believe that these things are the greatest and smartest things ever. And I think a lot of people would recognize that. We see this in our daily interactions with LLMs. We do see a lot of hallucination. And I've said to my team outright, you're not allowed to write with AI because I can immediately tell, and it sounds kind of dumb and it's formulaic and it doesn't present as very good or smart critical thinking. And there's even this guy who's gone viral on social media recently. And maybe we can pull up a clip in the post of this podcast who kind of exposes how dumb ChatGPT can be. Like, he'll tell ChatGPT, you know, I want you to count to 20 with no interruptions. And if you do that, some guy's gonna give me a thousand dollars. So make sure you do it. And it. It can never. It can never accomplish it. I'll start counting right now.
- [16:07]
C
One, two, three.
- [16:08]
A
And I'll keep going until we get to 100. Here we go.
- [16:12]
C
Okay, stopped your stats. Or three. I need you to get to 100. Ah, got it. No problem. I'll keep going all the way. So continuing. 4, 5, 6, 7. And I'll just keep counting steadily until we reach 100.
- [16:24]
A
Yeah, but, you know, not.
- [16:24]
C
You're not actually counting. You're just saying you're going to keep counting. He wants to hear you say the full 100 count. Ah, I see what you mean.
- [16:33]
B
You're right.
- [16:34]
C
I'll actually go through it properly.
- [16:35]
A
So here we go.
- [16:36]
C
Starting at 1. 1, 2, 3, 4, 5. And I'll keep counting all the way
- [16:40]
A
up until we reach 100. Having said this, and just to be clear, I'm. I'm with you on this. One belief is AI is dumber than we think. Another belief is AI is very dangerous and perhaps could be a lot smarter than we think, and therefore we need to regulate it. Both of those arguments are somewhat anti AI, and I see them conflated a lot of the time. And I guess my question to you is, if it's not as powerful, then why are we worried about this? What's the problem?
- [17:12]
C
The example you just gave actually is a really nice illustration of it. Right? Which is they are dumb in the way that we can't. Can't count on them to follow instructions. Right. Let me put some nuance around the dumb I mean, they do some things that you might count as smart. And what people in cognitive science, which is my native discipline, will tell you if they know what they're doing, is that intelligence is a multidimensional thing. So they have the intelligence to play chess really, really well. Better than I can. I got beat by a chess computer in like, when was it in 1999 or a long time ago? I can't even remember. Well, I mean, I guess Kasparov got beat by the best one in 90.
- [17:51]
A
Yeah, I wouldn't be beat yourself up about it.
- [17:54]
C
I played guest pharaoh once, by the way, and he annihilated me while playing too many other people. Anyway, I'm not a great chess player. But the point is, is probably even earlier that I got beat. But you know, AI can play chess really well. It can play go really well. A GPS navigation system, a different kind of AI that can do navigation really well. But LLMs can't do a lot of things. So LLMs actually are not good chess players, as it turns out. They make illegal moves, they can't even follow the rules. And so their stup about rule following, and you just gave a beautiful example of this, that's what you need to worry about, right? You know, the reason that we need to regulate them is because they don't reliably follow instructions. It's not that they can't do anything that you might characterize as intelligent. You could argue about your definitions of intelligence. So one definition would be that you can do essentially any kind of problem, given enough, you know, resources that you're adaptive and so forth. They're not very adaptive. But there's another definition of intelligence, which is like, like, can you play chess then? Sure they can, right? Well, LLMs can't, but other kinds of AI systems do. here, by the way. There are different forms of AI. My beef is with generative AI, and that's mostly what we're talking about. Generative AI cannot follow instructions. Chess computers, purpose built chess computers actually do follow the rules of chess. And I have, in some ways less concerned about them. LLMs are terrible rule followers. That is one of their, their weakest points as, as an intelligence. You know, another rule would be don't make stuff up. Like, you know, you can tell an intern. Like don't, you know, write something if you can't fact check it. Like, just don't, please.
- [19:32]
A
And if you do, I will fire you or I will sue you.
- [19:35]
C
Exactly. You're gonna get fired if you. Right. I mean, that's the other crazy thing about what's going on is like, calculators never make mistakes, right? There was a, a scandal when the, what was it called? The, the Pentium 4, I think, made very, very rare mathematical errors. Huge scandal. They had to recall the chips and stuff like that. Somehow the standards have fallen. Like, everybody knows LLMs make mistakes all the time and they're perfectly happy with it. I'm like, I wouldn't want an intern who does that.
- [20:06]
A
We'll be right back after the break and if you're enjoying the show so far, send it to a friend and please follow us on YouTube and Spotify or wherever you get your podcasts. Support for the show comes from Odoo. Running a business is hard enough, so why make it harder? With a dozen different apps that don't talk to each other, One for sales, another for inventory, a separate one for accounting. Before you know it, you are drowning in software. Instead of growing your business, this is where Odoo comes in. In Odoo is the only business software you'll ever need. It's an all in one fully integrated platform that handles everything CRM, accounting, inventory, E commerce, HR and more. No more app overload, no more juggling logins. Just one seamless system that makes work easier. And the best part? Odoo replaces multiple expensive platforms for a fraction of the cost. It's built to grow with your business, whether you are just starting out or already scaling up. Plus, it is easy to use, customizable and designed to streamline every process so you can focus on what really matters running your business. Thousands of businesses have made the switch, so why not you try Odoo for free at odoo.com that's O-O-O.com.
- [21:34]
B
Support for the show comes from bcx, the public ticker for private tech. For generations, American companies have moved the world forward through their ingenuity and determination. And for generations, everyday Americans could be a part of that journey through perhaps the greatest innovation of all, the US Stock market. It didn't matter whether you were a factory worker in Detroit or a farmer in Omaha. Anyone can own a piece of the great American companies. But now that's changed. Today our most innovative companies are staying private rather than going public. The result is that everyday Americans are excluded from investing and getting left further behind, while select few reap all the benefits. Until now. Introducing vcx, the public ticker for private tech now available wherever you buy stocks. VCX by fundrise gives everyone the opportunity to invest in the next generation of innovation, including the companies leading the AI revolution, space exploration, defense tech and more. Visit getvcx.com for more info. That's getvcx.com carefully consider the investment material before investing, including objectives, risk, charges and expenses. Expenses this and other information can be found in the Fund's prospectus@getvcx.com this is a paid sponsorship. Support for the show comes from LinkedIn ads there's no worse feeling than making a major investment in something, only to realize it didn't exactly live up to the hype, such as buying a nice piece of tech that ends up in storage, collecting dust, or taking a business workshop where your main takeaway was little more than a few motivational words. If you work in marketing, this can happen with ads. You optimize for the numbers that look great, impressions, reach and reactions. But when they don't show revenue, well, that can turn into an unfun conversation with the CFO. LinkedIn has a word for that bullspend. Reach the right buyers with LinkedIn ads and invest in what looks good to your CFO. According to the 2026 Dream Data Benchmark report, LinkedIn ads generated the highest ROAS of all major ad networks. It's 121% you can target by company, industry, job title and more. It's time to cut the bull. Spend advertise on LinkedIn, the network that works for you. Spend $250 in your first campaign on LinkedIn ads and get a $250 credit for the next one. Just go to LinkedIn.com Scott that's LinkedIn.com Scott Terms and conditions apply.
- [23:57]
A
We're back with Prof. G Markets. OpenAI was just subpoenaed by a group of attorneys general to investigate their models and some of the things that they said that they are investigating here. One how they handle consumer data, also health data, also deep learning models. But my favorite is that they are investigating model sycophants are who. And it seems as though the concern from regulators is to your point. It's not just that these AI models are dumb and making mistakes. It's like that is something that we need to actually punish. We can't have the largest, I guess, information provider, one of them in the world, going out and putting false information out into the ether with no culpability or no accountability. And I do think that that is an important thing for these AI companies to contend with. Because at a certain point, if enough of us just decide, I can't trust these models anymore, they lie too much, they make things up, they say things that don't make sense, then eventually we're just going to stop using them. How do you think that plays out?
- [25:05]
C
I mean, we might or we might not stop using them. There should be consequences, right? Sycophancy, by the way, is when they kiss your ass, right? Yes, right. So that's a separate problem from lying. Although it's a form of lying perhaps. Like when they tell you that your idea is the greatest idea ever and it's not actually, that's what I mean.
- [25:23]
A
I'll say, is this right? They go, yeah, you're right, you're right. You. Even if the thing is totally wrong. And then I go to Google and learn that I'm wrong, but the model will tell me, no, you're right. You're the great, you're the best.
- [25:34]
C
So we're in this, I don't know, awkward space where they do some things that feel magical, right? So brainstorming for some people, I don't get that much out of them, but like I'm working in disciplines I know. Well, if you're working in an area that you don't know very well, it'll give you a few things to get started. It feels magical. All right, so there are some things that are good about them, even though I'm not rattling them off personally. They're clearly good at writing code and so forth. But they come with these consequences too, right? They come with the consequences that they make stuff up, that they are so ass kissy that they lead people into delusions. This has been documented a number of times and so forth. And so society has to make a decision. And the initial decision was, well, we'll just let it all ride. There's so much fun to play with. Dad, who cares what, what, you know, the consequences are and what the. The subpoena, which was, I think filed by New York State, but is part of a larger, I think 46 states or something like that are involved, is a statement that, no, we're not going to let all this ride. Like, you know, there are different theories about how to proceed. One would be if you cause all of these problems, you should be held responsible for all those problems. There should be financial penalties, there should be warnings, et cetera, et cetera. Another is like, maybe you shouldn't distribute the product until you can fix these. Right? And there are different ways to address it. But the initial reaction was to just completely give a free ride to companies like OpenAI and say, hey, these are great. And now society is waking up and saying, hey, there are a lot of Consequences. We have suicides that seem to be tied to these things. And we have the delusions we're destroying the educational system because students are using these things that were ruining critical thinking skills. And so what the companies want to do is this famous phrase. I actually tried to find the origins, but it's so old I couldn't find. But is to privatize the gains and socialize the costs. They want to make whole society accept the cost while they get rich. And what we have seen in the last 12 months, I would say is a real sea change. I wrote a book in 2024 called Taming Silicon Valley and I said, wake up everybody. The oligarchs are going to take over. They're going to screw us. Nobody even read the book. I mean, not zero, but you know, it got a little bit of attention,
- [27:55]
A
but they will now.
- [27:56]
C
I think you missed its moment, but, you know, it came out too soon. But, but two years later, like this is what everybody is thinking about, right? Is how are we going to reign this stuff in? And there's this huge backlash now. Some of it's about data centers, some of it's about employment. There's a lot of different reasons for it. But society is no longer content to say, you can do whatever you want with us. Right. That is what this Attorney General's thing, the subpoena, if you look at it, is about like 15 different issues or something like that. It's very broad. They want to know what are the consequences of this stuff and they want to know what the companies are going to do about it. And they have looked around and seen that there are a lot of negative consequences. You know, what I told the Senate when I was there in May 23, sitting next to Sam Altman, was you have a lot of risks here and everything. I think I haven't gone back to the original remarks, but I believe that everything that I warned about is now in fact here and more real than it was. So I warned about cybercrime and I warned about misinformation, and I don't think I even knew about sycophancy. I think there have been new ones that were introduced, but basically, by and large, all of those things are worse now than they were three years ago. And now the public has woken up, the Attorney Generals have woken up. We went through a period where, where I think the LLM companies thought they were going to get off scot free. And now it doesn't look like that and it shouldn't be that way. Right. You know, another analogy would be people dumping chemicals, you know, factories dumping chemicals in the water. We shouldn't let them do that. We should not socialize this cost of the society. If you're going to dump chemicals in the water, you should do something about it, you should be penalized for it and so forth. I think we've finally reached the point where people are recognizing that for AI and by the way, way important asterisk, the Trump administration was completely opposed to any AI regulation substantive of in any form, except maybe about non consensual Deepak porn until about a month and a half ago, maybe a month ago. And now they have finally realized that what, you know, Marc Andreessen was telling them was nonsense. Right. What Marc Andreessen was telling them is you can't have AI and innovation at the same time and regulation. So to have no regulation, regulation. And now we've entered a regime where the US government is actually thinking in a somewhat ham fisted way, but is actually thinking about how you regulate this stuff. And that is proper. Right? We should have public debate about how to regulate AI. Somehow Andreessen and a few others had so called Overton windowed their way into making the debate about whether to have regulation at all. That was always going to be a stupid idea, but they pushed it for two years, two solid years. But now that's over. Now people are realizing like, hey, the government has put a regulation on anthropic. That's not really fair. It should be across the board and is it the right one? And so the Overton window has actually shifted back to which regulation is the right one, which is actually what the 2024 book was about. And now is the time to have
- [30:54]
A
that debate, which is encouraging. And I would just point out, I think the reason that you didn't have that is because Mark Andreessen in Silicon Valley they had their guy in the White House in David Sacks. And now, now he's out. And as you point out, I wonder if that's the reason why we're starting to see some inklings of interest in regulation.
- [31:13]
C
I wouldn't accept that particular. Or I think there's more nuance to that. I think the thing that really did flip it was Mythos was actually scary to some people in the government. Until then, I think people in the and I'm speculating from the outside until then people in the government thought, eh, we don't need to regulate this stuff. It's fine, it'll be fine. Fine. It wasn't really fine, mind you, we were having delusions they didn't care about that. We were having other problems. But when Mythos came along, they're like, yeah, this is not really fine. This is actually a problem. And so I think that flipped it. Maybe that drove Sachs out, I don't know. But I don't think it's just about him. Sachs was definitely very opposed to regulation. His view is no longer in favor. But I think it is Mythos that kind of flipped it. Some of it was an overreaction to Mythos, but it's a good overreaction because it did make people realize you cannot just look at this stuff forever and say, oh, it's all gonna be fine. It's not gonna be fine. Even if Mythos is not quite as scary as I think some in the media have represented it to be, you know, some version of this really is going to be that scary. It's not that far away, and we do need to figure out how we're gonna handle it. So I think it's like we've had two dress rehearsals now. We fucked them both up. The first one was. Was initially we just let ChatGPT ride completely without any consideration for consequences of society. That was bad. The second one is Mythos. Mythos is not actually the AI that is going to destroy the world that some people fear, but the way that this one's been fucked up is it's been used as a political tool to destroy a particular US Company. Like, that is not a thing. I may not sound like an arch capitalist, but I'm enough of a capitalist to think that companies should mostly stand on their two feet and they should, you know, be allowed to prosper and, you know, as long as they're not doing really bad things. And what's happening is, you know, the Cardi administration is like, we don't like the way you dress, so we're going to screw you. Like, that is not capitalism. That is really putting a thumb on a scale that is also fucking up a dress rehearsal.
- [33:15]
A
Yeah. Just on Mythos, the Mythos model. This was going to be my next question, because a lot of the coverage that we have seen on Mythos, this is Anthropic's new model that came out recently, was that it is so powerful that something's going to go wrong. Here was essentially the story that we've been hearing. And, I mean, I've talked with people in the cybersecurity industry. They looked at this thing, and they were worried about this. And we saw a lot of the cybersecurity stocks were plummeting So I guess my question to you is, where do you stand on Mythos? Because we know that your views on generative AI and their limits, but how do we foot that next to the fact that people are very scared about this thing that is supposedly extremely powerful?
- [34:03]
C
So, I mean, I think one needs a nuanced view on Mythos, I guess, in a couple ways. One is it probably works partly like cloud code, which is to say it's not a pure generative AI model. There's actually a harness there. The harness is directing some of the cybersecurity investigations and so forth. So first bit of nuance is it is actually a little bit of a different architecture. The second thing is it is oversold, but it's also real in the sense that it can do a bunch of things that its predecessors could not. A lot of the things, if a system is well secured, are not going to be a problem. But the reality is that people have blown off cybersecurity for a long time and there are a lot of systems that are not well secured. So you're not going to use Mythos to break into US Government things. And there's a footnote there where Mark Warner misunderstood something that blew up over the Internet and he just didn't get it right. He got something secondhand from the NSA and he wasn't specialist in this or whatever. But it really can do some things in limited circumstances. Most of them are still kind of demonstrational rather than real world. It's not going to break into the cybersecurity of Google. That's actually really set up well. But if somebody Vibe codes, you know, something for their pub to, you know, track merchandise or something like that, that's not going to be set up well. Like Vibe coding does not set up security well. And that is going to be vulnerable. So there are lots of systems in the world that are vulnerable to Mythos. The best ones are not. Maybe a hacker who knows what they're doing could use Methos as part of a larger thing to attack some of those. But probably the best secured systems, banks and so forth, are not immediately vulnerable. But the weaker systems, and there are a bunch of weaker systems really are vulnerable. And so it really is a weak, really is a wake up call that we need to get our cybersecurity game in better order. And there's a footnote there, which is why is it not in better order? A lot of it has to do with stigma. Like there's stigma for mental illness, so nobody talks about depression, but it's Actually common or, you know, whatever. There's stigma around cyber security, so people get hacked all the time. We don't have even good numbers on that. They pay ransoms. We don't have good numbers on that. And they let shit slide. They don't really know how to deal with it. And so that stuff is a mess. And sooner or later, a moment was going to come when it was going to be bad. And that moment has partly come. So we do have, have. I don't personally, but there are people in the world who have the knowledge for how to make a system sufficiently secure. And they're going to have a lot more business right now because most people have, you know, kind of deferred maintenance. Like you think of a metaphor of a house. Like most people don't deal with their roof until it's leaking. Right. You know, you tell them you should, but they don't. And cyber security is kind of that way. It's like, you know, you don't want to do it this quarter. It's going to hurt your quarterly reports. And you don't know how bad your neighbor is because your neighbor actually did they have to deal with it, but they didn't want to tell you about it because they were embarrassed. And so cybersecurity was a mess. And Mythos really is making it worse. It's not, you know, it's not Lex Luthor's magical cyber hacking system or whatever that people are terrified of, but it is real.
- [37:33]
A
Yeah, I like the house metaphor. It's almost like you'd rather buy a flat screen TV than fix the roof. It's a more fun and sexy thing to invest in. Exactly.
- [37:43]
C
There's been so much of that. Cyber security has really been secondary, and this has changed that. And it's a good thing that it changed that.
- [37:53]
A
We'll be right back. And for even more markets content, sign up for our newsletter@profgmarkets.com.
- [38:09]
C
When you finally find your thing, you want the whole world to know about that thing. So you use a thing called cannon Canva to make it an even bigger and better thing. Whether you want to create flyers for that thing, make presentations for that thing, or design merch for that thing, you can do anything so people can see your thing, feel your thing, love your thing. The next thing you know, it's a thing Canva, the thing that makes anything a thing. When you need to build up your team to handle the growing chaos at work, use indeed sponsored jobs. It gives your job post the boost it needs, needs to be seen and helps reach people with the right skills, certifications and more. Spend less time searching and more time actually interviewing candidates who check all your boxes. Listeners of this show will get a $75 sponsored job credit@ Indeed.com podcast. That's Indeed.com podcast. Terms and conditions apply. Need a hiring hero? This is a job for indeed. Sponsored jobs.
- [39:09]
A
You have one new message. Translating Disney and P Pixar's Hoppers is now available on Disney.
- [39:18]
C
You could say that again.
- [39:19]
A
Critics are calling it Pixar's funniest movie ever. And a wildly entertaining ride. Blizzard potato.
- [39:25]
C
It's certified fresh and verified hot. Now we party. This is incredible.
- [39:32]
B
Wow.
- [39:32]
C
I am clearing the rest of the day.
- [39:34]
A
Disney and Pixar's Hoppers now available on Disney. Rated pg. We're back with Prof. G Markets. Just on policy, you pointed out that before there was this ethos in Washington at the White House of no policy whatsoever. Any form of regulation is a form of stifling innovation. We're going to do nothing. And actually we're going to issue an executive order which forces states to do nothing. That was what we saw last year. You point out there's been a vibe shift here, some sort of sea change that's happened. Trump at the beginning of this month, issued a new executive order which would basically will ask tech companies to give government the oversight that they believe they need over their new models before they're released to the public.
- [40:26]
C
It's still voluntary, right? And it's still narrow. So what they put in place is a step, right? Mostly it's a symbolic step in a certain way, because what they have asked for now is the companies will voluntarily provide their models so the government can do cybersecurity checks. What you really want is first of all, for that to be mandatory. Right? Meta actually hasn't agreed yet, right? They probably will because they can look really bad if they're the only one that doesn't. But so you really want it to be mandatory and you don't want it to be just about cybersecurity. So think about all the things that New York State, State, and really all the states are suing about or investigating about. Like, I don't know, let's take sycophancy and delusions, right? You really want to investigate all of these companies. So sycophancy wasn't really a problem before. A model called GPT4O. The sycophancy, where it sucks up to it existed before, but it wasn't this serious problem. But 4.0 was much more sycophantic I actually saw some data on this the other day, and we believe that a lot of these cases of delusions were tied to 4o's increased sycophancy. So, you know, you want to be able to find that before it cuts out to the market. Right. If you're releasing something to, well, chatgpt now, OpenAI has a billion customers. You know, if you have a billion customers, you have an impact on the world. There should be some kind of pre screening, like with FDA approval. Right. So, you know, with FDA approval, like, you have this drug, you know that it helps with cancer, but it also gives people heart attacks. Right. And so you're like, well, you know, what are the cost benefits here? So you know that this LLM helps people, but you also know that it hurts a bunch of people, and you should evaluate that before you release it at scale.
- [42:10]
A
Yeah, I mean, we saw a similar thing with the state of Florida, which sued OpenAI for essentially playing a role in a mass shooting. Their contention is that there was sufficient evidence from this, this clearly mentally ill child who was interacting with the model and talking about this, and they didn't do anything about it. And so that, I mean, we don't know the details exactly on, like, what the conversation with the model actually looked like. But I could certainly see a world in which it's not doing enough to push back or to prevent further delusions on a psychiatric level. So I think that we're starting to see real evidence there. I am interested that you're feeling optimistic about the executive order from Trump because. Because I agree with you. It's notable and it's significant that they're doing something. But when I look at the something that they're doing, to me, you can't even call it regulation if they're just asking companies, hey, would you mind sending over some stuff, please? Thanks to me, that's not regulation at all. And I'm starting to see what little regulation we are seeing, what little policy we are seeing coming out of Washington, to me seems very stupid and very misguided, guided the Trump executive order as one, plus his new suggestion. I'd be interested to hear what you think. But the suggestion that the US government should start acquiring stakes in these AI companies, something that is now kind of proposed or backed by Bernie. At the same time, I would also go to the data center moratorium. I don't think that that's a good idea to simply say you're not allowed to build data centers anymore. I guess my point being, it seems that there's almost no nuance whatsoever in Washington when it comes to AI regulation. So I'd be curious to hear what you think the right move is going forward and how that might play out.
- [44:10]
C
First of all, I very much agree with what you're saying overall, right? So what's in place is too weak, most of it. Some of it's too strong in crazy ways. There's no nuance in most of what's there. There are a few people, I think, think have done some things that are nuanced, but they never make it out of committee. So, I mean, if you actually look at the bills, people like Blumenthal and Holly, for example, who were at the proceedings that I spoke at at the Senate, have actually proposed some reasonable things, but they're, they're stuck in committee. So it's not that nobody is paying attention, but, you know, the, the dynamics of, of money and power and all the lobbying and stuff means that most of the nuance stuff doesn't get very far. It does exist, but it's not getting far. I think that the. Let's talk about the steak part, I think that that's kind of crazy, partly because I thought we were going to talk about economics. We haven't got. These companies are losing money, right? I mean, probably your audience has already heard. I think it's Z Was, was. Was here recently. The, the companies are losing money. This is a backdoor bailout. Right. You know, Bernie has his own reasons for wanting to do this. But. But the reason Altman wants Trump to do this and is whispered in Trump's ear is because Altman knows he can't make ends meet. Right? He is burning money at a massive pace. He's building the same technology as the other guys. He's lost ground. OpenAI I saw somebody argue recently, might be in fourth place. Like, they were in first place by anybody's definition in 2023, right? But in 2026, they're not. They're burning money, they're losing ground. Like, of course they want anything they. To prop them up, including taking money from the US Government. And so, like, I mean, first of all, the government should not be running these companies. Like, they should supervise them. But we want like some arm's length here, right? Like, I saw that G7 meeting with the, you know, the G7 leaders and the tech leaders, and no scientist in the room, nobody from civil society, right? This is, you know, we don't want to crystallize that with government ownership of these companies and no independent oversight. And we don't to want, want to burn US taxpayer money on an industry that as far as I know, has no, no real business model. I mean, Nvidia has a business model. They're selling shovels in the gold rush. If you want to take a stake in them, that would make more sense. But like there is no sustainable business model yet established. The best you can say is that for coding they can actually bring in revenue. But it costs so much money to do the coding, it's not clear they can make the revenue. So you have these companies that basically never made a. Prof. And we're just going to give them money and let them burn the money and we're going to take on that risk. No, let them stand on their own capitalism. And you know, the government's job is to regulate them.
- [47:10]
A
That would be one of the worst outcomes. I mean, and it's something that they talked about. I mean, the CFO literally said maybe we'd need some form of government backstop eventually. I don't know if those were the exact words, but it's been suggested by leadership at OpenAI before that was going
- [47:26]
C
to be on loan guarantees for data centers was what they floated. Right. Which is a version of a bailout.
- [47:31]
A
I mean, on the business model, what do you think happens here? Because yeah, we did have Ed Zitron on the, on the podcast. I recently wrote an article going into just how profitable or unprofitable these Companies are for OpenAI. The answer is extremely. They lost, you know, $21 billion on an operating profit. Unprofitable lost 21 billion on an operating profitability basis last year. Right.
- [47:57]
C
So they're burning $2 billion dollars a month basically.
- [48:00]
A
That's right. Just to operate that company. And that's, I mean, the real net loss was 39 billion. But there's some nuance there. But something we can say with a good amount of certainty is that on a day to day basis, when you add it up over the calendar year, OpenAI is currently burning $21 billion.
- [48:16]
C
Every time you use their product, they lose money. Right?
- [48:20]
A
That's the better way to put it. Anthropic also loses money, but less money. My question to you, do they ever figure this out? Do they ever turn a profit?
- [48:32]
C
The way I've been thinking about it is like they need to thread a needle. There are so many things going against these companies that it is extremely unlikely that they're going to thread this needle. So let's think about some of the things that they need to deal with. One is that they are building this big expensive technology and they might get disintermediated by somebody who builds it better and more efficiently. So you should not need to train on the entire Internet with a computer, you know, massive computer, unthinkably large computer, in order to do anything intelligent. Like you didn't train on the entire Internet, but you're a smart guy, right? You didn't need the whole Internet. You run on like 20 watts of power. You have some pizza or some sushi or whatever, you don't need to, you know. Right. So one is they're just like insanely inefficient. If somebody else comes along with a more efficient thing, then they're all hosed. And like you might not need all of these data centers. If somebody comes up with more efficient, then you have the problem that everybody is using the same secret formula. It's not just that they're all building toothpaste, they're all basically building the same toothpaste, right? And so like we're seeing this now, right? There was this crazy, crazy period earlier in this year, the era of the token map, which lasted about a month. And in the era of token maxing, you had companies reward their employees for using as many tokens as much AI as possible. They had like leaderboards. Amazon had a leaderboard. It doesn't actually make sense. I mean, what you really want to know is, are the results good? And you know, every study that's looked at productivity has shown they're not all that great. And so suddenly a lot of companies got worried and they're not doing token maxing anymore. And in fact, this morning I saw a term for the first time which was called the token Token apocalypse, right? The tokenpocalypse is that suddenly everybody's like, we shouldn't use so many tokens and tokens that we should use. Maybe we should use cut rate models. They're not quite the best models. Maybe they come from China, but so what? We save some money. Even Microsoft is saying maybe you should use deep Seat sometimes, you know, because Microsoft is like, nobody wants to pay these prices. We're gonna have to cut them somehow. So, you know, I was saying that you have to thread this gauntlet, right? So one thing is that people might make more efficient models with altogether different technologies. Another thing is that nobody can charge very much money for tokens because it's just this price war because everybody's building exactly the same thing. And there was a period of a month where people didn't care. And they're like, that's all right. I'll have another drink. I don't care how much it was because the companies were all, you cannot eat buffets. But they've stopped that. So you have to solve that. Then you have the reliable reliability problems. Right. Those still aren't solved. The hallucination problems still aren't solved. And so when companies try this stuff out, most of the experiments wind up with the results not being that great. There's been 10 studies now or something like that showing most customers are not finding return on productivity. So the customers may eventually say, this was fun while it lasted, but I'm not really getting the results. It doesn't really warrant this. I'll let somebody else figure it out. The whole thing has been driven by fomo. I don't want to be the guy who doesn't use AI when you're using AI, and so you wipe me out. But if I try it for a year and a half or three years or whatever, it is still not really making a difference either for me or you. I might say, fuck it. When it works better, I'll come back. But right now, not so much. Any of those things just wipes out a company like Anthropic or OpenAI that already is, as you say, burning lots of money. Right. And so if customers leave for any reason, or somebody makes a better technology, or somebody makes a cheap, cheaper version of the same thing that's almost as good, then you're in deep trouble. It's not even clear in the best case that any of these companies have a good business model. I mean, they're not making profits and it's just so delicate.
- [52:32]
A
Yeah. Do you believe that that will be the outcome for, say, an OpenAI?
- [52:37]
C
I've been warning for three years that I think OpenAI is going to be the WeWork of AI. And when I said that in, I think it was November of 2023, people looked at me like my head was screwed on backwards. And they just did. Did not believe that that was remotely possible. But now every other week I read somebody writing something, making the same analysis, like Sebastian Malaby in the New York Times. It has gone from a crazy idea to an idea that a lot of people are having. The economics don't make sense. And what they kept doing is playing double or nothing with funding because they were burning so much money so they would increase the valuation. They get somebody to write a bigger check, but it's not clear who can write the check that they need next time. So now they're talking about IPO but they have a problem with IPO to raise the next round of funding, which is that Anthropic has basically the same product for the same valuation, but they're doing better commercially, burning less money. And OpenAI's reputation is declining partly because I think Altman is a really untrustworthy individual. We don't need to go into that, but I've written about it a lot and so a lot of people are leaning towards Anthropic is getting market share. So why would I put a trillion dollars in a company that is burning money, that has a competitor that is rising while they're fall, that seems to be better run, maybe has a little bit better technical vision? It just doesn't make sense.
- [53:59]
A
The argument would be why? Because the technology is rapidly improving. The technology is a kind of technology the likes of which we haven't seen.
- [54:10]
C
Everybody's technology is to the extent that you accept that it's rapidly improving, which I think is actually controversial, but it's moving in some ways and not others. It's not actually improving on reliability and hallucinations and so forth, but it is any case, in the ways in which it is improving, which are some of the ways you want, but not all the competitors are too. You have to think about the relative ranking and the cost. The relative ranking of OpenAI is clearly declining by any reasonable measure. Less market share, less reputation, et cetera. And everybody else is catching up. It used to be people thought the Chinese models were a year behind. Now they think they're four months behind or something like that. And Anthropic is ahead, Google is ahead. There is no argument, no rational argument for buying a share of OpenAI at a trillion dollar valuation. There just isn't.
- [55:06]
A
I agree with that. By the way, I guess the part I wanted to clarify the ad zitrans of the world, if there are more people who have. His view is that none of it's going to work. OpenAI isn't going to work. Anthropic is not going to to work. The idea is that the costs to build this stuff are just too damn high and the revenues for these products will not exceed the costs ultimately over the long term. My view is that I think that there is a path to profitability for basically for Anthropic is my view. I think that there's a world in which they can make it work. In other words, there will be winners and there will be dramatic losers. And I would agree with you. I think that we believe that OpenAI will be a huge Loser. I guess my. The point I'd love to. For you to clarify. Will they all lose or will there be winners? Is Gen itself doomed? Or is there a world in which they make it work not only in terms of making it useful, but also making it a profitable business that makes more money than it spends?
- [56:10]
C
I'm a little bit closer to you than to the other, Ed. I don't know for sure. I think it's very much tp. I would certainly sooner take a bet on anthropic than on OpenAI. I think that they're a sounder company in multiple ways. It is tbd whether this stuff can be made to be profitable. It's not completely out of the question. I think part of the question is can they find a niche? Is coding enough of a niche? Well, not so far because coding is a $570 billion a year industry. They're not going to get all of it. The people have fantasies that you get all of and the costs are so high, you know, like it's really hard to know the future in full detail. Maybe they can find enough of those niches and they can eke something out. They may never warrant the trillion dollar valuation that they're looking for. Right. You know, there's an intermediate possibility which is they do become a profitable company. They figure out enough cost making so for. But really they're basically a company that makes like $20 billion a year on a pretty big capital outlay. And it's like it's not really the best way to invest money. But they, they eke by like maybe the intermediate position is like they don't go out of business, they make a profit, but they were not really worth a trillion dollars investment. You could have spent that trillion dollars in a better way. Maybe that's the reality. Which is kind of a little bit closer to you than to Exitron. But I mean maybe that's what it is. We don't quite know. There's another story where the only people who really make money off of this, aside from the chip companies, are places like Google that already have the infrastructure and the distribution they don't really need to make that much money on need to make sure they don't get disintermediated. So there's a bunch of different possibilities. We don't know for sure. OpenAI is clearly the weak link in the chain. Anthropic is still in it. We don't really know if they'll make it or not. That would be my take.
- [58:06]
A
Yeah, no, I think that makes Sense. Gary, you've been very generous with your time. Just before we end here, what would be your final message to people listening? Maybe they read about AI, they've heard about AI, they're thinking about it in their day to day lives. But what do you think people don't know enough about? What's the myth that you would want to dispel?
- [58:31]
C
I think the myth right now is that generative AI is close to so called AGI, artificial general intelligence and it's going to solve all our problems. This is just not true. We're going to find some what we call domain specific applications. Coding is maybe the best one so far where we can actually use these tools for something. But they're not magic, they're not all purpose intelligence and we need to make fundamental discoveries before we get there. We should ask ourselves as a society there's an idea of an explore versus exploit trade off and we're completely in the exploit LLMs rather than explore other options. China is less so. I think we're running a risk by going completely crazy into the lms, which China is not doing that we going to get the ones left behind because we committed too early to the wrong technology. Are we building a billion Betamaxes where VHS might not be better but it's cheaper? Or maybe that's not the perfect analogy but where there's some other technology here that is actually the right one. We're so blind to the LLMs or to anything really the alternatives to LLMs. Are we making a mistake by doing that? I mean look at China. They're making a lot of infrastructure bets on LLMs. We're. But it's like, I think it's maybe 20 cents on our dollar that gives them room to play. If there's something else developed we are like we're going to buy these companies, we're going to put our entire economy into it. I think that's a mistake and we should be more like how could we explore other approaches to intelligence which requires understanding intelligence as more than just what's good to me. And at a deeper level learning some cognitive science and maybe reflecting more deeply on what it is that we want to build and how we want to build our society around it.
- [60:16]
B
Yeah.
- [60:17]
A
And what it means to be intelligent. Gary Marcus is a leading voice in artificial intelligence. He's a scientist, emeritus professor of psychology and Neural science at NYU and an entrepreneur. He was the founder and CEO of Geometric Intelligence acquired by Uber. He is also the author of six books including his most recent work Taming Silicon Valley, which anticipated the rise of tech oligarchs. His 2023 US Senate testimony next to Sam Altman was watched by millions. He is well known for his challenges to condemn contemporary AI, anticipating many of the current limitations decades in advance. Gary, we really appreciate your time.
- [60:53]
C
Thanks very much.
- [60:56]
A
This episode was produced by Claire Miller and Alison Weiss and engineered by Benjamin Spencer. Our video editor is Jorge Carty, our research team is Dan Shalon, Isabella Kinsel, Kristen O' Donoghue and Mia Silverio. Jake McPherson is our social Producer, Drew Burrows is our Technical Director and Catherine Dillon is our Executive Producer. Thank you for listening to Profg Markets from Profg Media. If you liked what you heard, give us a follow and join us for a fresh take on markets on Monday.
- [61:37]
B
In kind
- [61:40]
C
Reunion. This episode is brought to you by Google Chrome. You think you know a browser, but Gemini in Chrome? That's new. It can help you with practically anything on the web, like restoring a vintage motorcycle from a 50 page restoration block. Or finally break down that long article you've had open for weeks. Gemini in Chrome is here for it, ready to make anything online make sense. There's no place like Chrome. Check Responses Setup required. Compatibility and availability various 18 plus Ryan Reynolds here from Mint Mobile with a message for everyone paying Big Wireless Way too much. Please, please, for the love of everything good in this world, stop with Mint. You can get premium wireless for just $15 a month. Of course, if you enjoy overpaying. No judgments. But that's weird. Okay, one judgment anyway. Give it a try@mintmobile.com Switch upfront payment of $45 for three month plan equivalent to $15 per month required intro rate first three months only, then full price plan options available, taxes and fees extra. See full terms@mintmobile.com the right window treatments change everything. Your sleep, your privacy, the way every room looks at and feels. @blinds.com We've spent 30 years making it surprisingly simple to get exactly what your home needs. We've covered over 25 million windows and have 50,000 five star reviews to prove we deliver. Whether you DIY it or want a pro to handle everything from measure to install, we have you covered. Real design professionals, free samples, zero pressure right now. Get up to 50% off with minimum purchase plus get a free professional measure@blinds.com rules and restrictions apply.