B (5:38)

A lot of when I'm reading the book and thinking about is doing the better job is that it relies on better incentive structures. It relies on kind of a good faith approach to wanting to do a better job and not just make money. Right.

A (5:52)

I mean, one thing about the word community is it's people who all have skin in the game and they've got problems that are in common, and so they actually really want to solve those problems. They're not just spouting off. So if you can get groups of people who are actually genuinely trying to solve some problems, then you can generally find something that they all believe, or they can educate themselves about what's going on and they can take actions. And if we have enough of that going on. Well, that's what the enlightenment was about. There were sort of these social experiments of people trying to live in different ways, and some of them were pretty good and some of them were pretty lousy, but we copied the good ones when we were realized we weren't doing so hot.

B (6:34)

The Internet, we have the new royal postal system, right, where everybody's connected, everybody can talk to each other. But you sort of mentioned this as like a false start. What happened?

A (6:44)

Well, there's two obvious things that are wrong. One is that connecting everyone to everyone is actually probably not a great idea. You want to connect to people who have similar problems, similar situations, so that you can figure out what to do. Right. Random comments by somebody who shares nothing in common with you is probably not going to be terribly useful, although it might be amusing. And people drift off into this sort of entertainment alarmist mode. That's one thing. And interestingly, if you look at Facebook, which doesn't have a community, everybody's a friend. Yeah, sure, tell me another one. Right. But it turns out that the groups that they have now, those are the Facebook groups. If you look at the people that are in there, they're people that share physical reality too, almost always. So, like, if you have a basketball team where you go, you'll have a group, but you don't have groups typically that don't have a lot of real shared interests. So people have taken that structure that ignores the notion of community and shared interest and twisted it to make it like that. Communication should be meant for communities. And you could be part of several communities, but they're people of shared interest. The other thing that they do in the pursuit of money is they allowed influencers to grow. And so there are these very loud voices. You make more money if you're more angry. And when we do experiments, and we do experiments across the whole country, we find that people don't know anything about other people. All they know about are the influencers. So if the Democrats, what do the Republicans believe? Well, the thing they do is they pick out the crazy guys on the right and they say, well, that's a Republican. And the Republicans do the same thing about the Democrats. And they're both almost exactly equally wrong, which is to say they're mostly wrong, almost completely. And even on issues like abortion and gun control and things where, you know, there'd be a clear divide, there isn't. There's great consensus in the country and then there's the crazy people who are like going to blow everything up unless they get their way. So the obvious thing is you want to like knock down the loud voices and you want to provoke some sort of civility. And we actually have built an open source platform. It's at deliberation IO if you want to look at it. The code's all there. A lot of experiments. And what we find is that if you take something like polarization, things like gun control, things like that, if people use this, it's sort of like X but with visualization. And there's a little AI in there that says, I hear people saying this, but it doesn't contribute any content. It's just sort of moderating. What you find is people get depolarized really fairly dramatically over very short periods of time. So we're using this for town halls and things like that. And other people have taken it and they're using it for education, you know, have a classroom discussion, et cetera, et cetera. So I think it's possible to build things for communities where the AI is not contributing content. It's not telling you what to do. It's just trying to be like a mediator. It's trying to sort of cause people to focus on the problem, behave themselves just a little bit, because people want to do this. They have skin in the game if they're in that community. So it turns out it works. Surprise.

B (10:08)

You know, people want to go along with the sort of group consensus, right? There's a Urge to be part of the majority. But with fractured communities, it's like, which majority are you part of?

A (10:21)

When people talk about that, they say it's like, oh, it's like this human weakness to want to go along with majority. No, another reason to want to go along with majority is if you're going to actually do something, like you're going to make an investment in a new road or a new school, you need a lot of people all on the same boat. Right. So the reason to find consensus is that that is what enables you to change your life for the better.

B (10:45)

Now we have the new technology you mentioned, like generative AI that makes its own stories. Do you feel hopeful or worried about that?

A (10:54)

I spend most of my time in Silicon Valley and people are either doomers AI is going to rise up and kill us all, or they're lotus eaters. Oh, we'll never have to work again. Which actually sounds pretty bad to me because I like to think about things and I don't think that's likely either of those. I do think that people will use these tools in both good ways and bad ways, just like they use every other tool. And we ought to be ready for it because these are pretty powerful tools. The fact that you can spin up 30,000 bots in a fraction of a second to make it look like the majority believe X or to manipulate a financial market is pretty scary. And our current systems don't acknowledge that possibility because it wasn't possible before. So we need to think really clearly about that. And in particular, there's a couple of things. Like, I just read a paper yesterday showing that the doubling rate for AI competence, so this is how good a little AI agent will be, is three and a half months. That means it's 10 times better at the end of a year. That's like, okay, we're on a rocket ship, boys and girls, we better ask where that rocket is and who's driving. And who's driving, Right? Yeah, well, hopefully it's our communities. One of the things we've been trained to do over the last few centuries or millennia even is say, oh, we've got to have a single person in charge, because there's no way to make a decision quickly that involves everybody. Well, yes, there is. Go take a look at this deliberation IO thing. Not that it's the answer to everything, but it might give you inspiration that it is possible to do that. And the other thing is that people, including me, are not so smart. We're not good at this logic thing. We're not so good at consistency. If you ask a person a couple questions at one time and then you ask them two weeks later, they only agree about 70% of the time with themselves. Okay, well what are we going to do with that? Well, we know what to do with noisy signals. If you put lots of them together, you can get things that are pretty consistent and so good. The other thing about the AI systems I think is very promising is if you think about a lot of the problems in the world like the legal system to move or how do you get bureaucracies to move. You know, these are all systems that were designed two centuries ago. They're mostly patterned after armies. Guy at the top tells you what to do, right? And that's just a recipe for being uncreative, for being trapped in this sort of funny hell that we all are when we think about big corporations and we have possibility of breaking out of that and really being much more community oriented, much more shared wisdom. And you know, people say, oh yeah, yeah, great, stupid dreamer, I don't need to listen. So let me just tell people. There is a thing called the Uniformed Law Commission. It's a volunteer organization. A bunch of lawyers from different states get together and say, hey, if you adopt a kid in this state, it doesn't apply in the next state. That seems completely broken, what are we going to do? And they come up with example proposed laws that they take back to their states that process purely volunteer, purely distributed from all 50 states, accounts for about 10% of all the law in the United States today. It's been going since 1870. It's why you can buy things interstate. It's why if you're married in one state, it applies in the other state. It's the thing that sort of smoothed it all together, made it work on a larger scale. But it's not a top down thing at all. It's not the federal government. It's a bunch of people saying, I see a problem, what can we do about it? And the incentive for the lawyers is of course they get to show up in their home state, say hey, I'm part of this great thing and here's an idea. And maybe they make a little more money from that, but not from the thing itself.

B (15:00)

And this distributed sort of volunteer contribution you mentioned in the book, but it looks like open source software to developer.

A (15:06)

It'S just like it, very, very similar to that. Or maybe Wikipedia or maybe.

B (15:12)

Yeah, with open source software there's always some decision maker at the top, whether it's a Benevolent dictator for life or a governance board that makes the final decisions. Is that a necessary part of this?

A (15:27)

Some open source software? Each person who has building a system and deploying system makes the decision they're going to have these parameters and they're not going to put that part in. And maybe it work and maybe it doesn't. So a lot of it doesn't have that decision maker. We work with itf, which is the group that defines the Internet worldwide. And you know, there's a thing very much like the Uniform Law Commission, which is you get together and you talk about it and people say, well, what about this? And you try and reach consensus among all the people that are concerned about this before it becomes the law of the land. Law of the land. It doesn't mean you have to do it, it just means that you're not going to be compatible with other people, which is pretty strong motivation for doing it. Just like the Uniform Law Commission, the guy at the top is just a facilitator that says you should go talk to George or Mary or something like that.

B (16:20)

But I mean they take it to a senator or somebody who is then the decision maker who puts it in play or they don't. Right. There is some governance board, there are some governance.

A (16:29)

I mean the thing with the Law Commission is that at this point it's well known enough that you bring it back to the whole leg, not just the head of the Senate. Head of the Senate will hear about it and may be able to block it, but so will all the other senators and they may give them hell for that. And if you think about it also one of the other characteristics, and there's been a lot of research about this, is a community is defined by common interests. Okay, so if I point out something that's costing you guys money, then you'll go, hmm, maybe we better think about that one. Right. So it turns out that if you're focused on things that really matter in a day to day case and are very broadly applicable, people actually move, they change their opinions, they do things. That's not a very sexy thing for the news to report on, but it happens.

B (17:20)

Gender AI models are sort of statistical aggregations of their training data set, right. That all of the material in there, big squishy, stochastic parrots, all that do we lose a little bit of that community coherence when we tap on the AI models?

A (17:36)

One of the major problems with the AI models is that they took all the stories that were on the web and they like stuck them in one place so they're not sorted by community, and you can prompt them to do a little bit of that. But let me give you an example of something that I thought was really interesting. I was on a panel with the CTOs of five of the largest corporations and we were talking about AI and stuff like that and what they had to tell their CEO, right? But one of the things they let slip was, oh yeah, and we've built AI buddies for everybody in the corporation. And it turned out all five of them had. AI Buddy is just a little vanilla AI that runs locally that's read the manual that you didn't read, that read the newsletter that you don't pay attention to, that pays attention to what other people in the organization are doing and can answer questions about or remind you about things that are going on in, in your organization that you're working in. And it's cheap, it's easy, all that sort of stuff. It may not be perfect, it isn't perfect, but it makes you much more aware of the context, much more coordinated with other people without actually telling you what the answer is. It's just telling you what other people are doing. That's a wonderful sort of way of reinforcing community. It's just making sure that everybody's in the loop in that way and you know who to go talk to if you have a question, right?

B (19:02)

Yeah. I mean, for a modern CTO to keep track of all of the tech flowing their company, it's very hard. Like, talk to a company recently that does AI summaries of commits made into the software base and sort of rolls it up based on how detailed you need it to be. So something like that is sort of like gives you good context. But I'm sure anybody listening to this is going to be like, well, AI's hallucinate.

A (19:29)

Well, that's one of the hard things is you've got this tool which is really great in some ways because it can look at a lot of data and give you something, but it's stochastic. It doesn't really necessarily. So you need to have judgment in there. And I've noticed on your podcast you've got a number of people who point out that not having that reality check at the end can lead to bad things. I talk to boards of directors with some frequency and, you know, thing they're honestly interested in is what happens when the AI screws up and we have made a mistake with a million customers or we've done something that hits the front page of the news or whatever.

B (20:08)

And who do we fire?

A (20:09)

Oh, my God. Yeah. And with this squishy AI technology, this is going to happen. But think about the AI buddies. It was okay if the AI buddy says something sort of stupid, because it's there to just sort of remind you about things. It's not there to tell you what to do. And we have a project that I'm very proud of and I think is exemplary. We sort of got together with Consumer Reports. So people probably know Consumer Reports. They, you subscribe, you give them a couple bucks. They used to produce a magazine that had testing of consumer products. Now it's all online, but they still do the testing of consumer products so that you can buy stuff that's good and works. Right? That's the idea. And what we're going to do is we're going to build AI buddies for everybody. And the project name is called loyalagents.org so you can look at that. And what it is, is it points out that, yeah, you need authentication, you need authorization, those sorts of things. But there's also legal things, because when you have an AI that's representing what you're doing, and this could be in a company, too, if it screws up, you could be on the hook, or the person you're dealing with could be on the hook. And so you need to have something in there that talks about a legal checker. Like, I was at a meeting of 50 of the leading lawyers in Silicon Valley, and I said, no, no, you can't use an LLM to decide if it's going to be a legal problem or not. You need something deterministic so that you can point at the law, right, and say, yep, that one's okay and that one's not. And there's this concept of fiducia, which is something you should pay attention to. So doctors are supposed to represent you, Right? Not them. Lawyers are supposed to do the same. Finance people sometimes do that. But it's this idea of you hiring a professional to represent you. And in many cases, what you want is you want an AI to represent your intent. But how do you know it does that? Because, you know, it's made up of all this stuff. And if you talk to it for a little while, it forgets who you are. There's all sorts of stuff because it's a little squishy. Squishy. And can be moved. And so that's one of the main things we're doing, is trying to figure out how you can actually make AI do what you want. And only the things that you want, and there are solutions. One is a way of prompting it that gets the intent of your preferences across correctly.

B (22:38)

So that'd be a sort of context engineering.

A (22:40)

Yeah, it's a context. So we all know about mcp, presumably. Right. So this is human context protocol, communication protocol. And basically what it is, it's a way of saying, well, here's a request, but here's the intent, the human intent that gives you the legal context for doing this. Don't screw around with that. This one's important. Right. And that's important also, particularly because if you've got agents talking to agents talking to agents, it's really easy to lose the thread. And they do, and they come up with their own thread, which is really bad sometimes. So you have to be able to say, okay, the point of this is to pay a reasonable price near the bottom, but it has to bloody work and it has to bloody get here in two weeks. Right. Something like that. That's really the thing you need to have happen. And then it can have some degrees of freedom to work within that, but you have to be able to set that current AIs are sort of okay for that. But one of the things is how do you keep reminding them so that they don't run off the rails? Right.

B (23:44)

And how do you absolutely forbid something and sort of programmatically forbid every forbidden action?

A (23:51)

Yeah. If you talk to the lawyers, the way they think about it is you have to have a deterministic system that surrounds, that looks at every action that the AI does, says, here's the inputs, there's outputs. We're going to essentially have like an expert system, because law is basically an expert system. And we're going to check if this goes into a red area. And if it does, then we kick it out and look at it. Right. I think that it may be possible to do this internally, where essentially you construct prompts that are like that and you have things where there's the same AI or some other AI acts as a judge that goes in and says, okay, does that fit the prompt? Right. And they're pretty lightweight ways of doing that sort of thing. So it's not crazy to think about and remember that these things are getting exponentially faster in terms of amount of quote, unquote intelligence per dollar.

B (24:42)

Yeah. LLM on LLM evaluations are. They're pretty good, but they're still at the top end, about 80% aligned with human intent. I don't know if your lawyer friends would take that one into court.

A (24:54)

Well, 80% is probably not good enough. But like when you buy something with a credit card, they assume that there's a 3, 4, 5% fraud rate in there and they buy insurance. They have some fairly hard things on the outside, which is, you know, you can't do more than this amount and you can't do, et cetera. But those are pretty lightweight. You have the systems out there that are already doing things like that and they're doing it at scale every day in every country in the world and so forth. So there's real hope in doing this. You just have to remember that you want to have that audit trail, you want to have that check. You need to have that sort of intent prompt that is carried from agent to agent to agent and you need to check. I also like to tell like boards of directors and stuff that there needs to be an audit trail for what you're doing. Right. So this is just like when you spend money, you say, well, I spent this much money on that thing and I got it this day. Right. Simple computers are really good at this. But what that lets you do is when somebody comes up and says, are you biased against this group or are you committing fraud with that country? Right. If you have the audit trails, you can answer that question like right then and then. You don't get into this big kerfuffle with lawyers and courts or news people or stuff like that. It's not nothing. You need to really think about that.

B (26:15)

The audit trail stops once you get to the LLM itself. I think some folks anthropic is doing some efforts in trying to look into the brain of the LLM. But at that point it's like, well, how did you get this answer?

A (26:28)

I think that's not necessary. I think that's a whole sort of mistaken line of thought. I mean, let's take a judge and the judge listens to some evidence and makes a decision. Well, what went on in his brain? I don't know. You don't know? We do know that it's different when he's hungry and it's different on Monday than it is on Tuesday. Right, okay, but, but what you do have is you have, well, here were the facts and here was the decision. And when somebody thinks this is crazy, they take it to a higher level judge. Okay, well, that's how we do things. So we need to take our LLMs and we need to have some like little boundaries where, you know, we take it up to the higher level thing and take a better look at it. And you need to be able to answer questions about overall performance. If there's subtle little biases or things like that are going on that are not immediately obvious in a single operation. The one I'm most concerned about is not so much that, because I can see there's techniques for doing these things and so forth. It's the fact that you're likely to have LLMs calling LLMs, calling LLMs, calling LLMs to do something and you don't know three quarters of them. And how are you going to ensure that this is actually a valid chain of operations?

B (27:42)

We recently published an article on overall discomfort with some of the AI generated writing and art. But I want to get your take on how LLMs gen AI can actually connect to people. What's the positive spin?

A (27:58)

Well, let's go back the two things I already mentioned. So one is this AI agents thing. So if I ask my little LLM, assuming it's attached to the right sort of data, you know, who else is doing this sort of thing? It's pretty good. And in fact, obviously if you use Google Search or anything like that, you get a little AI thing all the time. And they're not bad. I mean you shouldn't bet your house on it. And so check the footnotes and maybe check them twice if it involves a fair amount of money. But the point is that's not bad. And what it's doing is it's taking human output like the web, just like Google originally search was, right. And making a summary of it that's easily digestible. So that's connecting people to other people. Now of course you've got little questions like this response, something that somebody paid to have promoted and blah blah, blah, which is why you probably want your own agent. I mean even if you have to like pay a little for it, right? You know, you really don't want it promoting so and so's product. Right. That type of a thing. And so this is going to be a big legal battle like Amazon and Perplexity I think are at it now over that. Right. So that's a way of connecting people and it's like this AI buddies type thing. A second thing is this causing groups of people to reflect together about what solutions are without permitting loud voices. So that's this deliberation IO stuff. So it's a lot like Twitter X something like that, but there's no influencers and you get to see a visualization of what everybody is saying. So you get a sense of where you are in the crowd and there's an AI that gives you Feedback about what everybody is saying. It's like, well, people seem to be saying this, but some people say that that type of a thing. So it's trying not to inject content. It's trying to give each person a sense of what's going on in a way that's otherwise sort of contextless. And you can go read the science, it's all there on deliberation. IO seems to work really pretty well and not be biased and blah, blah, even for things that are very contentious, like should we give guns to mentally ill people? But it turns out people behave pretty reasonably there. So there's ways of using them as information gatherers to connect. The AI buddies one is connecting to particular people. The deliberation one is getting people to connect to each other to make decisions. And then the third thing I think is really interesting, and this is in the book, is this sort of trying to think about what are the range of things that we can do and what's important. And the example I like best is we did something where we were just looking at all of the scientific publications in the world and noticing that there are places where nobody's focusing. And if you look at that, so you get this big web of science, right? And there's a hole in the middle. In fact, there's a bunch of holes. Those are the places where the big hits are going to be in the future. And interestingly, it works exactly the same with patents. So people patent all the stuff and they refer to each other. And there are places where people are blind somehow. Now we don't know what goes there, but we know that that's an area that nobody's binding. And maybe you ought to take a second look. So there's ways of assessing the global behavior of people to be able to ask how are we doing? What are we missing? And things like that. And that sort of bleeds over into things like protein folding and things like that. We have these huge complicated things. And what you want to ask is what are the tricks we're missing that type of a thing. So it's an innovation discovery engine, as it were. And those are the three things I talk about in the book. And I think all of them have the character that they don't remove the human from the conversation. They're about finding the things that other people say and meeting people. They're about hearing what other people are doing to try and think about things together or about reflecting on the broad range of people and what they're doing. But none of them are the AI is telling you what to do. So helping people build and shape their community is the idea.

B (32:11)

It is that time of the show again where we shout out somebody who came onto Stack Overflow, dropped some knowledge, shared some curiosity, and earned themselves a badge. So today we're shouting out a Populous Badge winner. Somebody who dropped an answer on a question that was so good it outscored the accepted answer. Congrats to Harshal for answering how to start search only when user stops typing. If you're curious about that, it'll be in the show Notes I'm Ryan Donovan. I edit the blog host the podcast. If you want to reach out to me, comments, concerns, questions, topics, et cetera, you can email me@podcastackoverflow.com and if you want to reach out to me directly, you can find me on LinkedIn.

A (32:51)

Hi, I'm Alex Petlin, faculty at Stanford and MIT, and we were just talking about my new book, Shared Wisdom. MIT Press and bookstore near you. Starting tomorrow actually, as it turns out. And you can find me on Wikipedia and LinkedIn and just about everywhere. Interested in helping people use AI in ways that are pro social and build communities.

B (33:15)

We love to hear it. Thank you for listening everyone and we'll talk to you next time.

A (33:35)

Sa.