← This View of Life
This View of Life
Evolving Prosocial AI: A conversation with Peter Fenton and David Sloan Wilson
01:04:03
Loading summary
Transcript52 lines
- [00:00]
A
You.
- [00:05]
B
Welcome Peter Fenton. I'm so happy to be recording this podcast with you, which I am titling Evolving Pro Social AI. So in the first place, welcome.
- [00:15]
A
Thank you. It's a genuine pleasure to be invited, truly to one of my intellectual heroes. Having read all your books over the years. Yeah, it's a real delight to be part of a conversation with you.
- [00:30]
B
Well, in the first place, thank you. Thank you very much. And AI is going to be occupy center stage here, but I want to lead up to it by hearing your story. Basically, let's humanize it by basically focusing on you as an individual. And just to give a little bit of my own introduction before you launch in, you got your BA at Stanford University in philosophy, plus your mba. And then you became a venture capitalist working as a general partner with Benchmark. And this gives you a ringside seat basically as both a participant and an observer of everything that's taking place in the world of AI. And you do this against really a strong intellectual background because you never lost your interest in philosophy. I met you through our mutual friend, Elliot Sober, the great philosopher of biology. And I know that you attend his classes faithfully whenever he comes to Stanford as a visiting faculty. And so you remain very engaged intellectually and then you bring that. And that's why I'm eager to talk with you about AI from basically a generalized Darwinism, multi level selection perspective, a perspective that is still greatly in the minority in the AI world. And so it's against that background. Could you please humanize this conversation in any way you like by just telling us how you wandered into this line of work and especially your intellectual interests?
- [02:17]
A
Well, thank you for that. History goes into the random chance events of being born and raised in the Silicon Valley. And my dad was the CEO for the bulk of my upbringing and gave me a window into the world of entrepreneurship and particularly that community around Stanford, and how there was something different here in the ecosystem which we'll talk about, I think, as we relate it to AI that allowed it to be maximally adaptive to new technologies. And I was lucky enough to go to Stanford as an undergrad and I was pretty avowedly anti commercial interests. And I found myself in rapture, as one does as an undergrad with the world of the mind, and in particular, I should say, in the world of philosophy, you have this perch from which you can dive into so many different fields and start to wrestle with the questions that are at the center of the work being done in those fields. And nowhere more than in my case, philosophy of Science and I, in my junior year I had a professor, Peter Godfrey Smith, who was my advisor, pull out this thin little book that was my version of poetry. And it was Elliot Sober's book on the philosophy of science. And within that book there were a number of topics which are still being unpacked around units of selection, questions of altruism, things that are at the core of the philosophy of evolutionary biology, which is his most recent book. And in that, in that time, which is in the early 90s, I was also interested in computational systems. And Tom Wasow was my other advisor who started the symbolic systems program at Stanford. And it was clear even then in the early 90s and it's very in vogue to say I was interested in neural networks before, you know, electricity. But in that case I was quite interested in neural networks and I took a class with David Rumelhart on optical neuronal processing and that the biological models were actually really effective at understanding computational models. And these worlds were not so separate. Even though if you'd listen to the dogma coming out of Marvin Minsky and the east coast regimes, I think there was a real hostility towards the use of neural networks because they just didn't have any of the simplicity and heuristic top down functions that were attractive at that time and quite frankly more effective at generating computational success.
- [05:13]
B
So those would be more special purpose.
- [05:15]
A
Algorithms then indeed more special purpose. And it was a sense of like, I don't know, the bubbling up of Californian mindset of loosely coupled systems collaborating together versus the command and control hierarchy that you find in the MIT culture.
- [05:34]
B
That's really interesting.
- [05:36]
A
So in that timeframe I was humbled, as one often is, thinking about becoming an academic like yourself, David, and that I just didn't believe that I could be one of those very few people that contribute like Elliot, and you have to advancing the field. In a sense I was as a student madly in love with the topics, but in a sense insecure maybe, maybe too much so that I could really have a major impact. And I think if people are listening to this and they're on the precipice of do you can continue down pursuing your academic path or do you face your setbacks and turn to the commercial path? There's not a right or wrong answer. I do think about that fork in the road in my life though. And having been lucky enough to take some graduate seminars with Elliot in the last couple years, I've been blown away that the quality, the density, that sort of rapture one feels even as an Undergrad can still be alive in a 50 year old's mind. So the road not traveled, for lack of a better term, remains kind of vivid. It in my mind and what's so compelling right now, and we'll talk about it today, is the collision of these worlds of evolutionary biology and the world of entrepreneurial technology, in particular artificial intelligence are, you know, now at the epicenter of, I think, not just, just the Silicon Valley, but quite frankly the human species.
- [07:07]
B
So yeah, that's exactly where we're heading. So that's, you know, but I'm really interested to hear the next step of how you actually did take the path that you've taken and also the culture of Silicon Valley. I think that you're going to be getting to that on your own. But I'm really eager to hear about that, you know, actually through the lens of this, this academic paradigm. So keep going. Basically, what happened next, what happened next.
- [07:36]
A
Is, you know, instead of getting a PhD, which I had probably thought about, dreamed about infleeting moments of potential hubris, I went to a management consulting firm, Bain, and well, it's one of those things. I had two job offers coming out. Nobody wanted to hire a philosophy major back in 1994, except for Bain. And I had an offer from Morgan Stanley and I believe sort of one from Goldman Sachs. And it's sort of appalling. And I look back at my life to think I would have gone from these high, you know, minded pursuits to debasing myself as a consultant for a couple years. But yeah, I mean, it had a positive. It turns out philosophy is a highly portable skill. One might argue that it's the effective talent of bs. I found it to be more applicable for deconstructing problems into logical components that you could then apply rigorous thinking against. And so two years at Bain and then I had, through my dad's work as an entrepreneur, been exposed to the world of venture capital. And I always thought that that would be the ultimate dream job, which is to work with entrepreneurs at the very beginning of the life of a company, serve as almost a co founder, a partner to these people on the wild adventure of building something from nothing. And the Silicon Valley, you know, you can take this crazy alchemy of one or two people can imagine something that is in less than a decade worth, you know, forget the money piece of it, billions and billions of dollars, but really, really have a meaningful impact on, on a human scale. And that gravity well in the Silicon Valley of just getting to entrepreneurship and working on it, to me, venture Capital was sort of the best, the best route other than founding your own company, which being a philosophy major, you're ill equipped to do much of anything.
- [09:36]
B
But I say that jokingly, except you can do everything.
- [09:39]
A
Yeah, you can do everything. And venture capital, I think it was a comment my dad made. He says, you seem to me to have your peaks and valleys, which is a broad way of saying my many deficiencies would be hidden in the path of venture capital and whatever strengths I did have would be amplified. And so. So to get into venture in that time, it was important, I felt, to work at a company, a startup, to really experience it firsthand before you start to make the decisions to invest in entrepreneurs and work with them over the years. And so in the mid-90s, I joined a startup. I was a product manager for about a year and a half, two years, and that was my launch pad into venture. And I've been doing that for, you know, since 98. So coming up on 30 years, I've probably worked with over, I think my two venture firms have worked firstly at Excel Partners and then went to Benchmark, where I've been for about 20 years. My guess is 30 plus companies. I don't do the numbers on this, but. But over half a dozen of them have gone public. I'd say over 10 of them have gone through a billion in revenue. And I've landed right now at the center of AI and that we have this ecosystem where the next big thing courses through the Silicon Valley's veins, I think more cleanly than anywhere else in the world. So I joined the venture business when the Internet boom was in its peak fever pitch in 1999. So sort of at the trail end of that. And then the next big wave was social, mobile, call it, you know, 2007, eight to about 2012. And I was lucky enough to be involved with the Series A investments at Uber, Snap, Instagram, Twitter, where at Benchmark we backed those companies at the very beginning. The next wave we went through in the industry was crypto. And in that case it really wasn't much of a Silicon Valley phenomenon, even though there are a number of companies like Coinbase based here. But these cycles in venture tend to be labeled with one technology. And that was sort of from like 2013 till about 2020, 2022. And then of course, the next big wave, and quite frankly, the biggest wave any of us have seen in our career started in, in earnest in 2022 at the launch of ChatGPT. You could speak to computers, you could have a conversation with them, you could imagine all these, you know, new possibilities, which is in our sense just at the beginning. And the Silicon Valley and venture capital in particular have proven over time to be the most responsive to these trends. And, and so the likelihood principle of like the next trillion dollar market cap company when a new technology segment comes online is high probability, as in like 90% likely in my experience to be based in the Silicon Valley. And we can get into why is that the case versus New York now.
- [12:47]
B
Let'S do that briefly and I've talked to other people about that, but I'm interested in your thoughts first because I think they relate back to evolutionary theory, believe it or not. But in your words, why is it that Silicon Valley is so cult structurally generative?
- [13:06]
A
Well, I think part of it is the evolutionary biology model of being adaptive as a system is highly applicable to the nature of the Silicon Valley. And there are certain things, I'll just point out a few variables that are conducive to the adaptability of the system. The capital resources are set up to seek the asymmetry of if you put a dollar in, you can only lose a dollar. If it works, you can get 1,000 or 10,000 and the best venture funds are generating 30 to 50 times. So there's a risk, comfortable capital ecosystem. There's a high degree of sharing of know how and expertise amongst the participants in the Silicon Valley.
- [13:54]
B
That's one of the. Keep going. That's one of the things.
- [13:56]
A
Yes. So that high degree of sharing, by the way, is also embodied in the laws. And so we can't enforce non competes in California, but in Boston and in Massachusetts and in the east coast, non competes are highly enforceable. So the idea of cross pollination, of transmission of the ideas and know how is deeply restricted outside of the Silicon Valley. And it's highly conducive here. There's the strong identity of the Silicon Valley of entrepreneurship. So our heroes are not financiers, our heroes are not movie stars. Our heroes are the founders of it's Jensen and Nvidia, it's Larry and Sergey at Google, it's Mark Zuckerberg. And so there's this aspirational sort of underlying ecosystem and on top of that you have a bunch of efficient flows of information around and this is the functioning of the ecosystem, hiring people, you know, the nature some systems are that I've worked in, feel like, okay, you want to keep everything closed inside of your network. The Silicon Valley has this default of if someone asked me, one of my competitors asked me about an employee, I'll tell them everything that's of value and use, because it's not, you know, to my standpoint, it's an ecosystem ethos, an ethic, a culture that is you support others because you never know, one day that may be you. Our heroes. People like Bill Campbell before he passed and Ron Conway are notorious for having multiple competing interests because they collectively just want the ecosystem to flourish. So as you unpack it and you pull in the threads or okay, what are the things that, that are also distinctive? There's this, this responsiveness of the Silicon Valley to disruption. And that to me is like most interesting, which is how is it that by 2022, when you launch ChatGPT, fast forward three years almost, you know, when it was November, I think of 22. And so fast forward three years, we probably have no less than 500 startups in the Silicon Valley that are premised on the existence of LLMs that would have been inconceivable to found in 2021. So the pace and the magnetic pole of the entrepreneurs towards the disruption forces is unlike anything I've seen. And part of is also is that we have an immigrant based. Most of the people we back, I'd say over 95% of the entrepreneurs the history of benchmark have come here, not, not just from, you know, the east coast, but from around the world. And, and that sense of like, you know, aggregating talent, all of this leads to, you know, I think you could apply your, your name was Tinberg's Principles.
- [16:51]
B
Of Tinbergen's four questions.
- [16:53]
A
Yeah, yeah. Function, history, mechanism and development, you know, around the Silicon Valley as the ecosystem. And you discover, I know there have been books written about just that the system is not yet been repeated anywhere else. And I think the mistake many people think is they apply one variable. Oh, it's access to capital or it's changing regulation. And I think they miss is that the degree to which it's adaptive as a system. And part of that is something that, you know, we, we as a system go through manic cycles. I don't know that this is this well publicized, but you know, our mania which we have right now, creates bubbles and bubbles are required in many ways to move innovation at a pace that would be severely restrained if we were more, more tempered. And so people always say like, is it a bubble Silicon Valley? And. But that's, yes, of course, that's the nature of what we do. And if it wasn't, then we wouldn't have this sort of like, it goes Back by the way to the gold rush in California, in San Francisco there's this sense of like levels of excitement that are totally unsustainable but all consuming. And there's an adage right now that a lot of our companies are sharing which is 996-9am to 9pm six days a week. The only other place in the world where that's really happening is in China. And I think in both cases there's a sense of like the totality and intensity of the opportunity is sublimating all the other things that are in your life. One of the non well documented things is that San Francisco, as much as it is a city, you arrive and you're like where's the Silicon Valley? You go to New York, you can find the financial district, you go to Hollywood, you see the Hollywood Hills. In the Silicon Valley you don't see any of that because it's all internalized. And there's this inside of the buildings, inside of the offices, inside of the cultures that are being created, a monomaniacal focus which comes at the cost by the way of other forms of diversity in the ecosystem. But the Silicon Valley just feels like it's possessed by itself as opposed to these externalized. Anyway, the outputs of all this is of course the products that we are delighted by and use. The depressive cycle which comes after the manic cycle will happen. I don't know if it'll happen 27, 28 or 29 but it's just the nature of this world. And I was there in the post bubble Internet from 2000, 2003 and 2004 and it felt like nuclear winter. It sort of felt like that after 2012 and 13 when social mobile had run its course in its primary sort of ascendancy. And we'll go through it again, but at this time and we'll get into the AI implications the stakes are higher.
- [19:52]
B
Because yeah, I would take this for a global. The bubble is a worldwide bubble indeed.
- [19:57]
A
And I think the effects like it's the single biggest capex in the history of humanity build out of AI infrastructure and it's pushing into our lives in a way where we co evolve with this technology but but at a scale and a pace we've never seen over a billion monthly active users now at ChatGPT and the pace with which it's penetrating our daily lives is unprecedented. So it raises a whole bunch of interesting questions around the health of that evolution.
- [20:34]
B
So Peter, now I want to channel a little evolution 101 here. This is what we both know but isn't really as widely known as it needs to be, which is that evolution doesn't make everything nice. So evolution often results in outcomes that benefit me, not you, us, not them, our short term welfare, not our long term welfare. And so that means that when we talk about cultural evolution and technological evolution, and most recently AI evolution, and I think artificial intelligence could equally, equally be called artificial evolution, we have all these evolutionary forces at work, but without some kind of regulation, to use a graded word, it can easily result in problems, not solutions. And cancer is kind of a metaphor for this. Cancer is evolution at lower scales becoming destructive at larger scales. That's the basic logic of multi level selection. And this is something which just pervades every subject in the natural world. It teaches us that nature is not just automatically harmonious. Special conditions are required for cooperation to evolve in nature. All aspects of the human experience, economic systems. So this is not restricted to AI, but AI has the same problems with this, the same dynamic, you might say, as everything that preceded it. And so what this means is that for any innovation then we have dysfunctional outcomes and basically dystrophic outcomes addition to more pro social outcomes. So something must be done. And what might that be? So I think that's the dilemma and I'd just love to hear you talk about that because I think you understand that as well as I do, but you're in a better position to apply it to the whole technology world before and after AI.
- [22:51]
A
Well, I appreciate the framing. I think that power of evolutionary biology to help us understand adaptive not just as sort of always aligned, but maladaptive versus adaptive towards the high functioning and flourishing of an ecosystem is a grand opportunity to take our thinking up a level to begin to understand the implications of these technologies. And we don't need to go far back in time to identify where you have pathologies that come out of the optimization of a system in a non regulated direction. Social media has left a scar on a generation of children and the algorithm made contact with humanity and the algorithm won. If you look at the side effects, and I know this has been popular with Jonathan Haidt's book on the anxious generation, but the idea that we co evolve with these technologies and that they become predatory in ways that are completely logical to. If you're trying to drive attention and attention is a vehicle from which you can monetize, then you prey on the human's weaknesses to fall into those traps. And I think that the consciousness we now have at the implications shows up and many of the famous executives we know who run those companies, don't let their children use those services. And they understand we've borne witness to how they can erode our sense of well being. So I think this idea that it's laissez faire it has been a bit of a Silicon Valley mantra, no regulation. And then you have the second model which is top down. My partner Bill Gurley has a post about a long talk he gave at this all in conference about why is the Silicon Valley Silicon Valley. And he gave you the number of miles between the Silicon Valley and Washington D.C. and said we built independent of the ecosystem of getting connected to regulatory capture. And we all know that regulatory capture has corroded major industries. Healthcare being the principal example of that. Which is why Covid tests cost 10 times in the U.S. what it costs in Europe. So there's these two competing models as you said, laissez faire on the one hand, top down on the other. But there's a third way and I think that the evolutionary biology gives us a really good example concretely in everything from multicellularity and the work that I'm not familiar with Lynn Marulis but understanding how does it that the you know our mitochondria formed a contract with the cell to be able to bacteria bonded and to form to a nucleus to have it so we have cellular life. And I think the same sort of sets of questions apply in our system and the simplification that I think is most potent and this is actually applicable in our companies is what you said which is that selflessness beats altruism within groups and altruism beats selflessness between groups. So how do we put evolutionary pressure on into the system where we have between group competition towards a higher functioning level. And I think that is one of those insights when you start to work on the alignment question of AI where the idea of I think naive idea of having top down regulations that are coming from a bureaucratic entity or it's okay, those people are the doomers. And then you, you ostracize that whole generation of people. And then there's been, you know, if you look at the press that people write about this they, they, they hyper focus on anyone who's trying to regulate is propagating violence against entrepreneurship because it's the sorts of things that have destroyed innovation in healthcare and in education and, and it's you know what's really behind it. And this is, this is the narrative. What's really behind that is the perverse incentives of the companies in the lead OpenAI anthropic Facebook, et cetera, to block innovation from the startups that can topple them. Is that part of it? I mean undeniably that's happened in regulatory capture in certain industries. On the other end of the spectrum you have laissez faire, maximum force innovation. And with that bottom up model you invite, you want to talk about cancer? I mean you invite this sort of entropy in a system where the self interest of any one party can destroy the overall organism. And we can look for concrete examples. Obviously we know that if you weaponize AI without any control on it, among other things, you have instrumental convergence, you have all these sort of the paperclip examples, all these stories about what happens when the system runs amok. And so we find ourselves in these, this polarity between the laissez faire crowd and then the regulatory crowd. And I think we have an opportunity to sort of go to his third, third approach which is bubbling up in, in the work you've been doing. And I think you don't need to look past the human body or you know, even an organ, you know, to see that there are multiple levels from which you can create alignment towards something that's functioning or if you don't have it, it's dysfunctional and it destroys itself. And so I think the question in front of the world for AI is really to say what can we take around alignment in highly functioning, both pro social systems but also flourishing ecosystems and map it into the AI project which is an organic ecosystem. It's not a single machine operating against an objective function. The most compelling thinker I've spoken to on this in our actual ecosystem doing work on this is Emmett Shear. Emmett started a company called softmax. I don't even know if it's a company, but he has a, he's already launched a competition, an alignment competition where you have teams and the way the teams come together, they're, they're, they're agents and they have multiple agents are on a group and I think this is a useful sidebar, if you bear with me. The group then competes to create a certain outcome. I believe it's, it's an object that requires multiple parts. And so you have multiple teams competing to create the largest output, which is the number of. And I think they're hearts, they're like there's a composite manufactured object. And what that shows you, if you have an individual in that team, an agent that's hyper optimizing their self interest, it destroys the group. And it's everything from the water slider example You've used in your history to, you know, playing Monopoly as teams versus playing monopolies against, you know, multiple players. And you have these emergent properties that come out of an aligned system around cooperation. And the implications from the programming standpoint are profound because the black box that is most of AI now has a new function that it can start to train against, which is being cooperative, being pro social, understanding in a way that, you know, simple self optimization can't get you. So it's just the beginning. I think we're starting to see that if you apply systemically the idea of a complex adaptive system as a system and not as a system of agents that are maximizing their own internal objective function, the outcomes are transcendent. They're elevating the entire project beyond what I think is sort of the extremes of laissez faire randomness, which is of course we know, going to breed self interest to the point of creating cancer, or this top down rigidity, which is ultimately something I think appropriately falls into the we can't even know the problems we're trying to align against, much less prescribe the solutions in a way that are effective. So setting up the AI ecosystem so that it is conscious of, insofar as the system can be conscious of the sorts of design principles and the aspects of a high functioning system requires the leadership of the industry. And I think it also requires deep understanding of highly comparable proxy systems in evolutionary biology. And we're just now starting those conversations, I think in an effective way.
- [31:30]
B
Wow. I mean, I'm so happy to have had you say that and to capture it in this recording. Peter, I have two points to make. One, I want to return to this idea and to elaborate on it, of major evolutionary transitions. And for the benefit of our audience, which is going to be very diverse. A major transition is one in which basically we have new units of functional organization. Basically, organisms evolve from groups becoming so cooperative that we now see the whole more than their parts. And this major transitions in individuality in the biological world accounts for in the first place, the origin of life. The first cells, first eukaryotic cells, multicellular organisms, social insect colonies, and then human humans as an ultrasocial species, and then carries on in cultural evolution from smaller groups to the mega societies of today. So there is like not just a theory, but a overarching panoramic cosmology worldview about how functional organization just expands and expands and expands. And according to Teilhard de Chardin, an early thinker on this is ultimately going to result in A planetary organism, Gaia and so on. And so what we, against that panoramic background, we can actually see AI as if appropriately conducted, could ascend that final run basically to the whole earth, basically as a cooperative, as a cooperative unit. And so I think that's what you were alluding to when you said a majority transition. And maybe you could reflect on that a little bit more and it will lead to the next point I want to make, which is that this perspective which becomes common sense and second nature for those who have been appropriately introduced to it, is still like 99% unknown in the economic world. And I wager in the tech world and even the most open minded, pro social people that are doing this, they just haven't encountered the corpus of ideas. And so just spreading the word basically is something that's so very important and can be like 50% of the solution all by itself.
- [34:10]
A
There's so many parallels. I want to dance around the topic a little bit, which is if you look at the idea of a complex adaptive system, the adaptiveness of a system is always at risk at the lower levels of that system, undermine the adaptiveness of the system. And one of the reasons the Silicon Valley is so great is that our big companies become powerful and then they get cancer and then they get ripped apart. If you think about this ecosystem, one of the things I was sort of depressed with in the late part of the last decade was the strength of the large cap network effect companies. Google, Facebook, now Meta. You can of course point to Amazon and Microsoft up in Seattle. And when the lower levels become maximally predatory and undermine the system's adaptability.
- [35:15]
B
It.
- [35:15]
A
Becomes vulnerable to the destruction of the adaptability of the overall system. And this seems like, okay, that's an abstract thought. What does that mean? We live and flourish in creative destruction in the Silicon Valley because the system is the Silicon Valley. It's not the companies. The companies are the organs, for lack of a better analogy. And when one starts to grow and become like cancer and suffocate innovation and becomes hierarchical, becomes all those things that the Silicon Valley has trained to hate, you know, we lose the magic of what makes it functioning. And so I think these questions of the ideal homo economicus, the individual operator maximizing their utility, make you blind to the actual phenomenon. And the actual phenomenon of the Silicon Valley as a system is relatively small units that are in competition with each other. But ultimately the emergent property is a cooperative system of high transfer of knowledge, high transfer of capital of resources, so that there's this transference and what I'm certain of, which is so interesting to think about this in Silicon Valley, is that in the next three to five years, there will be at least three companies that are founded that don't exist today that will become as big as Google and Facebook. And you think that's crazy. How could you say that? But that's the nature of the bubbling up of the innovation, is that. And they will devour and they will eat not the carcasses of these companies. Like you think of Cisco and Intel and others that were the prior generations in the Silicon Valley and how they've been eviscerated. And I think that that strength is the system's adaptability creates something that's much larger than any one lower level unit of selection. And I think this relates, this does map on to what's going on in AI. If you look at what's happened so far in AI, the people at OpenAI. Nick Turley is a friend. He was the product manager responsible for ChatGPT. And if you learn anything about evolution, it's three things, planned and unplanned variants. And I'm mapping it now to our world of entrepreneurship.
- [37:31]
B
Yeah, let me just flag planned and unplanned. It's really good. So cultural evolution will always have a bond component. It also needs a planned component. And so planned and unplanned. I just wanted to flag that as.
- [37:44]
A
Planned and unplanned is one variable. I'm struck by the degree to which the most interesting things are in the unplanned category. But that aside, secondly, you need selection pressure, which is you need to. Those traits have to matter in the sense of survive and reproduce. And then thirdly, you need a mechanism of inheritance. And these three things actually are the reason we have AI in the form we do today, is that OpenAI was messing around with a lot of unplanned variants. And John Shulman came to Nick Turley and said, I need data. We have this internal system here that was a chat system that would be a good way to have other people use outside of OpenAI. Which then led to launch of ChatGPT. And people all say that proof that it was unplanned is they called it ChatGPT. You wouldn't have called it ChatGPT if it was wizards who had planned the universe. It would have been a far more suggestive name. So that launches and it took off and first took off in Reddit Japan, in Japan, and then it caught the world by the imagination of the universe. And much of what's going on right now in the product evolution in AI, particularly on the consumer side, is unplanned. Meaning there's a pushing out of technology to see what people like and use and based on that it then gets inherited and then gets compounded. But the classic product management of a roadmap where you have linear plans against users, needs the proverbial jobs to be done framework for a product has been thrown out for the last three years and what instead is taking place is bubbling up of the capabilities of the models into the human's usage and then from there they start to capture inherit the things that have been of high function. But here's an Interesting comment. ChatGPT I've heard this, perhaps this is hearsay, but that another pressure for OpenAI to release ChatGPT was anthropic who come out of OpenAI was threatening to release a consumer product. So between group competition provoked the evolutionary system to start to be productive. One other thing I would say in this Dave, and I think it's somewhat relevant and it's relevant to the topic of alignment in the work in evolutionary biology that I was doing with Elliot. You discover that of course evolution's not one theory, it's a number of sub theories. Firstly, it's just things change over time, that's evolution. Secondly, you have in Darwin's theory common ancestry. You don't need to have that, but that's a core tenet. Thirdly, you have gradualism. Things don't happen without preadaptations. So it's not like all of a sudden a bird flew. No, there needed to be things that approximated a wing that created Bernoulli effect that eventually allowed for flight. Thirdly, you have branching of species which is they fork and they, so multiplication of species.
- [40:51]
B
Plus you have coalescence, coalescence also, but you got to keep going.
- [40:55]
A
Indeed you have coalescence. And then you have of course natural selection, which we just talked about. You know, evolution by natural selection is a special form of evolution. And so these, these, these theories all sort of give you a way to understand what's actually happening in the evolution of this technology. I will make a claim that people laugh at me for saying, but I, I'm increasingly of the view that AI is a speciation event, meaning I think it's entirely possible we look back, if we're here, which we probably won't be in a million years, and this non organic form of life and you could say is it life or not? I don't think it qualifies as life. That's a Fun debate to have. But when you have general artificial intelligence and it starts to, I think, look a whole lot like you would say, a species in terms of its ability.
- [41:51]
B
Its own life form, you're saying its own life form.
- [41:54]
A
And I do carry some fear that we may be the common ancestor to multiple forms of AI. One of the things about being a common ancestor is you're not around. So having said that, one of the things I took from the work that Elliot was doing is that directly applies to what's going on right now in our world is that when a new species comes online, 90% of the phenotypic change occurs in the first 10% of the species existence, which is such an interesting way to look at the way products evolve. So we can all pick up an iPhone today and say, kind of looks like the iPhone, as much as they market it as the most powerful iPhone ever that you could have got last year or the year before or the year before. But if you go back to the first 10% of the iPhones from 2007 to 2010, they looked really different and they didn't have video. The front button was changing on it. So we're in this period of time, which is why the stakes are. Are so high to, I think, think about to engage with the questions.
- [42:54]
B
Yeah, you get lock in, basically. So you get locked in.
- [42:58]
A
Look, look at, look at Instagram, look at the way that Twitter, look at these, these, the feedback loops of, of our first contact with the algorithms. We're preying on many of our weaknesses, and they have led to severe social implications that I think, you know, you take that with AI and say, okay, well, how do we start to make these systems mindful of the higher level of selection we're trying to achieve, which is a pro social, pro human form of intelligence?
- [43:24]
B
And that's where that's so. Yeah, there's so much to say, Peter. And one thing I want to say is that in your description of Silicon Valley, they are part of something larger than themselves. You said in several different times, you said there's this whole system, and the people within the system, they really do have the whole system in mind. So that's the, you know, the core identity and so on and so forth. And so in the case of AI is where all this stuff is happening, some plan, some unplanned, and then we're in the position of selecting, then that selection has to be based on basically, what are the, what are all the outcomes for systemically, basically, so that this doesn't become just a lower level selection pressure, it becomes a higher level selection pressure. That means that we're obligated, for example, to take all the externalities into account with all of these things. And so, and that's, that's what makes all the difference as to whether this is going to be a cancerous form of evolution or it's going to be one that leads to worldwide prosociality. And of course, currently we're very much in a cancerous mode. And I guess one question to ask is, are there some good examples of people within this system that are actually operating in pro social mode and are aware of this as you are? So what are some of the, like you might say some, some of the best case examples and worst case examples. So we could get the range of variation for this, including success stories in addition to.
- [45:13]
A
So a lot of what people believe. And I don't know that I'm channeling this as my thought, but it's just that seat belts came what, 30 years after the car. And the stakes seem much higher here because this is permeating in a way that we are co evolving with the technology at a pace that we've not yet seen in the history of any technology adoption cycle. If we think back and just step way back, it's at the level of the Industrial Revolution. And what we know about the Industrial Revolution in any of the major economies that went through the Industrial Revolution, except for the United Kingdom, they had civil war. So we're going to dislocate major parts of the workforce. And this is, I think, where the work of Peter Churchin is directly applicable in that there's a whole system effect of this technology dislocating the way that our civil society operates that's likely to lead to periods of disintegration in civil society. It's not at all lost on us that the first jobs to be really impacted, if you think about the way it hits the workforce, are recent grads with advanced degrees. And we know historically that if you launch a lot of recent grads with advanced degrees into the economy without work, especially if they're lawyers.
- [46:40]
B
Elite over production, for those familiar with Peter Turchin, elite overproduction.
- [46:47]
A
Yeah, indeed. And elite overproduction, which is sort of now amplified with the elimination. The inn is full, the seats are taken and half the seats are dummy taken by AI. We can look to the uk, which did a better job than other societies during the Industrial Revolution, to sort of figure out a way of more systemically addressing the immiseration of the population that was on the Receiving end of the negative effects of the industrial revolution. So there's a lot of topics on the table here. One of them is just the overall nature of our civil society is going to be impacted by this. But now if we move closer to the actual technology and the guardrails and the things you put in place, I think at the moment the work on trust and safety, safety and alignment and AI, we're trying to avoid the obvious two major negative effects. One is that it becomes an instrument for bad actors where you can use AI to develop weaponry, chemical, whatever. And then there's the other risk, which is the one that I think we all kind of in the back of our mind is the feels like the really scary risk, which is that the agentic runaway train that the AI starts to do things that we don't have an ability to see stop or, and you can't unplug it, so to speak. This idea of, well, just turn it off. It's like just turn the Internet off. So at the moment I think we have a trust and safety guardrail. And I do believe that the companies that are in this ecosystem are taking it seriously. But that's a different, those two sort of risks are different than what I would say is sort of a more evolved, pro social aligned ecosystem. And I think there we don't even know how to frame the problem except by taking models from evolutionary biology where you say adaptive as an organism, well, what does that mean? And immediately you start to push on that and you're going to get a fracture of what is it the objective function would be. So one example of these systems we know is that they go through reinforcement learning. They start to create synthetic data, they train themselves. If the objective function is profit maximization, then we're going to have outcomes that are at real odds with the high functioning of this technology alongside of humanity.
- [49:07]
B
So the objective function should be.
- [49:10]
A
Well, I think that that is, that is the debate, which is like, how do you have a. How do you allow for an objective function to be aligned but in a system that's inscrutable and the absence of observability inside of the AI, it's not like you could see, pull this lever, pull that lever. They're far more complicated than our understanding allows us to sort of break them into their piece parts. And so I think what's happening is something more like models from evolution, which is that you need to apply planned and unplanned selection, pressure and inheritance, but towards outcomes that we view as being aligned with humanity. And I think that the mindset of saying it's one thing to block them from doing bad stuff, which is sort of where I think the model is right now. And now we're discovering some emergent properties. We've all heard the stories of the, the people. You know, even south park makes fun of the fact that AI is particularly good at, you know, engaging or in long duration sessions to be increasingly sycophantic and increasingly aggravating delusions amongst the person who's interacting with it. Because the nature of the way these things are built as predictive machines is that they, they're set up so that they want to, they're engaging you to predict the next likely thing that to generate a deposit response. And is that a world model, is that coupled in aligned? No. And so we have a reactive our seatbelts that we're realizing we need. I think we're discovering as we always will, I think with new technologies too late. But more importantly, I don't think we have a systemic way of engaging the problem beyond this balkanization between laissez faire on the one hand and hyper regulation on the other. So I think it's ultimately a set of questions.
- [51:03]
B
Well, that middle ground, the third way between laissez faire and centralized planning is experiment, experiment, experiment. And so knowing that unforeseen consequences will abound. Life is like a folktale where you get to make a wish and end up regretting your wish. So that's going to happen. And unless we have ways of modeling it, detecting it, correcting it. And so there's your evolution variation, selected replication cycles that you're applying as fast as you possibly can. So Peter, as we wind up here, and this has been such a great conversation, how many people among your associates in the venture capitalist and AI Silicon Valley world have your degree of literacy about evolution?
- [51:56]
A
You know, I believe there are pockets of remarkable at the edge of our understanding thinking that are coursing through the company's discourse. But it's, I would say less than 100 people that I can imagine that really are preoccupied. There are certain thought leaders that stand out. Emmett Shear stands out in his company softmax. Dario and the team at Anthropic have been, I think, very articulate about the race to, he says, the race to the top, which is to attract the best researchers by being conscious about these questions. But as an ecosystem, I think we're struggling at the first step, which is what are the right questions. And it's one thing to say we have these competing factions and we do between Laissez faire and centralized planning. It's another to say that we have agreed on the core questions to wrestle with so that we can start to advance against that objective. And I think those questions come out of evolutionary biology models because it is a major evolutionary transition and we know what happens when there's dysregulation and we know what happens when there's overly rigid systems that aren't flexible. And so it's a precious few. But I actually have conviction that once people start to really think about the implication of technology, that it unlocks so much energy, because there are many in our ecosystem that are feeling like they're voiceless, powerless and in a sense helpless to do anything but pursue the objective function of make their company successful. And I think there's, there's a craving and aching for a engagement with the sort of questions that we put in front of us if we had those questions. Quite frankly, as I was there at the last little blip of social mobile, you know, we didn't, we didn't internalize and I was on the board of Twitter for eight years, but, but we had an instinct that there were anti social negative things going on. And by the way, the instinct was sort of graphically produced every day with the vitriol and scaling of antisocial behavior in online platforms. And there was one system that we all pointed to, and I think as an example, and I would hold this out there as an optimist claim, Wikipedia seems to have gotten it right.
- [54:27]
B
Yes.
- [54:29]
A
So there's a highly, I mean, adaptive as a system system. We know we have a model in online social and Reddit I think has done an okay job of this. But I'll tell you, like at Twitter, we never, we didn't learn from that in a way that was appropriately transferred into the Twitter.
- [54:47]
B
Let's dwell on this a little bit, Peter, because Wikipedia, I have a wonderful podcast with Anne Klen, who is one of the senior Wikipedia volunteers. And the podcast is titled Wikipedia as a Superorganism. And so, and so it is indeed. And there's other examples as well, I think, including in the whole world of open source. And I mean, one reason it works is that the objective function, the purpose, the raison d', etre, is not profit, it's a repository of knowledge or it's a software developing system that everyone can contribute to. So basically there's some goal, core design principle, one sense of identity and purpose, which is a overall systemic goal and that provides the selection pressure basically is to contribute to that. And the amount of structure that's built into Wikipedia. Could you just imagine all of the special interests that want to distort the knowledge of Wikipedia. It's truly like an organism that's being bombarded by diseases all the time, all the time. And if it doesn't have a strong immune system in order to keep that out, it's not going to succeed. And so it does. It has an anatomy and a physiology and all of that. So I think that. And it's not the only example. So the idea that we have some positive examples out there that we can understand and then emulate and expand upon I think is huge. And so thank you so much for mentioning Wikipedia.
- [56:26]
A
Well, I think that Wikipedia Jimmy Wales and the original ideas he was such a great example of unplanned variation because he originally had started it as a venture backed company to build a competitor to Encyclopedia Britannica. He failed and left the servers on and these Wikipedians took them over and they built this with the core, you know, the identity of a repository for the world's knowledge with a neutral point of view NPOV and it's a little bit like truth, you never find truth, you just get closer to it. So there's no neutral point of view, but there's something that's a little more neutral than that point of view. And so there's this sense of recursive trying to get better and that you're right all the mechanisms. And I think in a sense in an AI company it's critical that if you say our objective function has something that's pro social at its epicenter. And I do believe OpenAI the title of the company suggests that they care about that anthropic, they clearly care about it that are our major success stories. And if you're an employee thinking about one of those companies, you should interrogate what is the identity and purpose and understanding how that can sort of inherit then the immune system to prevent the bad outcomes that are the cancers that could. And this technology scale very quickly I think is one of those core questions. And I think if we get those questions front and center then we have a better, we have the opportunity for this to be as you say, in the superorganism of the planet. This technology seems of all the things we've brushed up against the one that has the most power in that regard. And we even talked about yet I was in China about four or five months ago, embodied AI and the idea that we're co evolving and we have a species which is going to be I think it's very reasonable to presume that by 2040 everyone's going to have some version of a robot, R2D2 like Android, like creature. And the pace with which China's building those things that are embodied, that are walking around, that would tutor you, that would give you friendship and companionship and the idea that this is a technology that you can put into a box and say it's over there. We are increasingly, as we have with language, by the way, co evolved with this technology and I think sort of being doing that consciously with a sense of like what, what is the identity and purpose gives us the best chance for it to be ultimately pro social and pro human. But you know, doing it without that, I think we are going to discover some very bad branches on that tree.
- [59:15]
B
Yeah, that's, that's so amazing. Well, Peter, this has been an amazing conversation and I think that one of the optimistic things that it holds is first of all, we need regulation. But that regulation can be basically self regulation and it can be something which is not imposed. And so therefore, if there must, first of all, I think there must be a consensus that of course we need regulation. An unregulated organism is a dead organism. And so first to get past the idea that we can somehow do without regulation and then to be the architects of that regulation so that it's not imposed by somebody top down, top down, basically. But so I look forward to this view of life, this worldview basically becoming more prevalent. What's required for that? Basically, because so much depends on just people seeing the world. Basically this worldview, this view of life, this worldview, making it more widespread within this community. What's your recommendation for that?
- [60:28]
A
My belief is the leaders that are at the front of the largest companies will discover either by the consequence of not doing it, the bad outcomes that get created, or by the benefit of doing it right, that makes it something we all aspire to. I'll give you a concrete example. I'm on the board of Sierra. Brett Taylor started that company two and a half years ago. They sell customer support agents and they're quickly evolving to be more than just customer support. And this is an agent that's interacting with your customer. And so in so far as that agent is left unchecked, unregulated, it can do a lot of very bad things. It can do some great things for the customer. Like this example of an agent in Air Canada had an agent give a customer a refund because they had a death in the family. The policies were violated. So Air Canada came back and told the customer that the agent went amok and asked for the money back and the agent was nowhere to be found. This is a terrible story because the customer sued Air Canada, Air Canada chose to litigate and the customer won because the agent gave them the money back. And so they. So the idea that you can do these things without any regulation will create stories that will abound of the run amok. And then the question is, okay, well, how do you start to inherit the attributes of a functioning pro social, pro aligned, you know, and I think we're going to. Because the contact these technologies are having with humans at scale, it will create the kinds of flywheel, like multiple experiments every day when these products are a billion times a month. If, in case of ChatGPT, that if we don't get this right, it'll be visible quickly. And I think that those implications then mean that you don't just talk about it, you have to embody it in the way you build your company. So my belief is that a handful of leaders, and I think Dario and Anthropic is an example of this. I believe Brett is doing this. And the other thing Brett and the founders did at Sierra was they donated a part of their company to retrain people who are going to be displaced from the workforce as part of the understanding the broad systemic implications of the technology. So like any transcendent moment of humanity, it's not one person, it's the system. But it requires agents within that system to be thinking about the whole and not their own self interest. And I think we, you in particular, but all of us, you know, amplifying those stories, they become the, they shape the narrative. And if we, if we also put some attention on where it goes wrong, we can understand the consequences of not focusing on it.
- [63:15]
B
Well, that's the perfect way to be to end. So thank you, Peter, and such a delight, you know, to take what's inside the ivory tower and bring it outside the ivory tower in a moment in history. This pivotal moment in history is the most important thing you could have done. So I'm so happy you did not get your PhD and end up stuck in some stupid turret of the ivory tower. So, anyhow, all great and so thanks so much. I look forward to sharing this far and wide.
- [63:54]
A
Thank you, David. It was a joy to be here.
- [63:56]
B
It.