
Loading summary
Andrew Jaffe
Hello everybody.
Marshall Po
This is Marshall Po. I'm the founder of the New Books Network, and if you're listening to this podcast on the New Books Network, I bet you like to read. I know that I do. That's why I founded the New Books Network. So as readers, we need to know what to read. And I have a podcast to recommend for you. That being the Proofread podcast, do you have a goal to read more this year? How about a goal to read more of what you love and less of what you don't? The Proofread Podcast is here to help you. Hosted by Casey and Tyler, two English professors and avid readers with busy lives, Proofread helps you decide what books are worth spending your precious time on and what books aren't. They have 15 minute episodes that give you everything you need to know about a book to decide if you should read it or skip it. They offer a brief synopsis, there's fun and witty commentary, and there are no spoilers and no sponsored reviews. Life's too short to read a bad book, so subscribe to the Proofread podcast today. And by the way, there's a new season coming soon.
Ad Voice 1
Toast the holidays in a new way and raise a glass of Rumchata, a delicious creamy blend of horchata with rum. Enjoy it over ice or in your coffee. Rumchata. Your holiday cocktails just got sweeter. Tap or click the banner for more. Drink responsibly. Caribbean rum with real dairy cream. Natural and artificial flavors. Alcohol 13.75% by volume 27.5 proof. Copyright 2025 Agave Loco Brands, Pojoquee, Wisconsin. All rights reserved.
Andrew Jaffe
So good, so good, so good.
Ad Voice 2
Give big, Save big with RAC Friday deals at Nordstrom Rack. For a limited time, take an extra 40% off red tag clearance for a total Savings up to 75% off. Save on gifts for everyone on your list from brands like Vince Cole, Haan, Sam Edelman and more. All sales final and restrictions apply. The best stuff goes fast, so bring your gift list and your wish list to your nearest Nordstrom Rack today.
Marshall Po
Welcome to the New Books Network.
Greg McNiff
Hello. Welcome to the New Books Network. I'm your host, Greg McNiff. I'm thrilled to be joined by Andrew Jaffe, the author of the Random How Models and Probability Help Us Make Sense of the Cosmos. The book was published by Yale University press, November of 2025. Andrew is a cosmologist, a member of Imperial Astrophysics, and the director of the Imperial center for Inference and Cosm. His work focuses on the history and evolution of the universe as a whole, with particular emphasis on the interpretation and analysis of cosmological data using principled Bayesian statistical techniques. And we are going to get into all those themes very soon with this interview. Just why I selected this book because it's just really fascinating in the number of topics it tackles and weaves together. Specifically, it focuses on the deepest questions in science. How we make sense of the universe shaped by probability and randomness, and what kind of models and how accurate these models are. Andrew ties physics, philosophy, and mathematics together in a way that's rigorous but still accessible. There are a few formulas in the book, and I know that may scare some people, but they're a very easy. Andrew walks through them and candidly, some of them are just beautiful the way they tie them together. I mean, it's amazing how simple these formulas are that address some of the larger questions in the universe. And frankly, the framework he provides offers real insight into how scientists evaluate arguments, question assumptions, and structure their models. It basically gives you a very nice introduction to the scientific method, which is a little more messy than many of us would suppose. So great read. Thrilled to have Andrew here. And with that, Andrew, thank you for joining me.
Andrew Jaffe
Greg, thank you so much for having me. Thank you for that beautiful introduction and yeah, I look forward to talking with you.
Greg McNiff
Perfect. Andrew, why did you write the Random Universe and who is the target reader?
Andrew Jaffe
Well, I wrote this book. It's sort of a funny story. I wanted to write a book for a long time. I wrote when I was younger. I wrote things that had nothing to do with science. I wrote for music magazines. I wrote for other kinds of things. I just liked writing, but I knew I wanted to be a scientist. And of course, we do a lot of writing as scientists, but it's a bit different when you're writing scientific papers. And several of my colleagues had written popular science books over the years, and I thought I could do something like that. But for years I didn't know what book to write. And I would complain to these friends, especially the ones who had written books before, and they just wanted me to stop bothering them. So one day I was coming home from an event in London with one of them, and I was in my Uber, he was in his. And he called me up and he said, here's the title of your book. And he said, the Random Universe. And it was kind of amazing. As soon as he gave me the title, I knew what the book was about. It was. It just tied up all of the things that I cared about in. In the Way I do science and the kind of science that I do. And it, the outline just sort of popped into my head immediately. Now to be fair, that was in 2013 and so it's still been a long time since then, since I wrote the book. But, but I, but I did finally get it done and we're here to talk about it today. It's a book intended for non scientists, for people who want to know not only about the science of cosmology, the science of the universe on the very largest scales, but also how we do cosmology and I think to, to, to a good extent how we do science in general. What are, what are the tools, the very abstract tools that we have to make really concrete? The tools of model building, the tools of probability, and you know, interpersonal communication in order to, to start with these vague ideas and end up with scientific theories that seem to fit the facts?
Greg McNiff
Yeah, I want to get into that and I want to start with someone who, I think your description of scientific theory is definitely a good description of him, warts and all, is Eddington, Arthur Eddington. And you begin the book really talking about the relationship between models and observation and a theory. And you quote Eddington, who says observation is not sufficient. We do not believe our eyes unless we are first convinced that what they appear to tell us is credible. Could you briefly talk about the relationship between a scientific theory and observation? As in which comes first? I think a lot of us feel like you collect the data and then form a theory, but that might not be the case, you know, particularly with cosmology. Whereas you point out everything, we don't actually see everything in real time.
Andrew Jaffe
It's even more basic than that. We all, every single day of our lives, in fact, really from when we were little babies, we have a model for the world. The only way you were able to find milk when you were on your first day in the world was because your body sort of had this built in mechanism, this model for how the world works and to know how to move your lips and move your limbs in order to find the milk. And eventually your brain figures out object permanence, the idea that there's actually stuff in the world that's out there and that it's not just this pictures on a screen that's sort of floating in front of you. All of that is a model for the world and you have to, you can't interpret the world without the model. Right. It doesn't work the other way around. And that's the same thing in science. You can Gather some data. You can have some ideas about the data, but it doesn't really make sense to you until you have a model. Now, sometimes that model is pretty simple and just the, you know, what did, what did Copernicus have to do? What did Galileo have to do? What did Kepler and then eventually Newton have to do? They had a model that they could figure out by looking at the sky. These moving dots, the planets and how. Something about how they moved around the sky. And you know, the naive, the most naive model that people had was that they were moving on the firmament of the sky and they were just moving around. They'd have to move back and forth. And it didn't really make a lot of sense. And so we then tried to interpret that in different ways that did better or worse. Copernicus tried to put the sun at the center and everything else moving around it, but in circles. And that's not as good a model as Ptolemy's. I didn't mention him. But, you know, he's one of the people who put the circles on the sky and the epicycles and really the way that we could account for all of the back and forth movement that we do see the planets make. And that was a great model. It fit the data incredibly well. Right. It just wasn't. It just didn't have a physical realization that made a lot of sense, but it, you know, it to our modern minds. But it worked very, very well. Copernicus's model worked much worse. It didn't predict the positions of the planets particularly well in the sky, but there was a beauty and a simplicity to it.
Greg McNiff
Yeah.
Andrew Jaffe
That people like Kepler and Galileo took up and then Newton to finally really make a model that, yes, you needed it, you needed the data to corroborate the model, but in order to interpret, in order to really make sense of that data at all, to really, to really put the fact that, you know, the. These things that look like they're all kind of the same distance away are actually at vastly different distances away. You need that model to begin with. You can't just start with the data and naturally have that bubble out. That's why it took so long for this seemingly very good model to finally fit.
Greg McNiff
And I want to drill more into that because you trace the, I guess, evolution of the model of the universe from Newton to Maxwell to Einstein and how revolutionary Einstein's. I don't want to say overturning Newton's model, but building on top of it. And I think Eddington contributed to that with his Confirmation of the, the bending of light. I forget the actual experiment.
Andrew Jaffe
Gravitational lensing.
Greg McNiff
Gravitational.
Andrew Jaffe
With clicks.
Greg McNiff
Yeah, totally. I know the World War I, it went a little off the track or something, but I want to get back to that. But Andrew, just one more definition, very broad. What is science? And I'll read in the book you quote, if I had to define science in a single sentence, I would say that science is the process of making inferences around the world from data.
Andrew Jaffe
Yeah. So we, we, we do. We do. It seems like we start with data. But what is so. What is so that's your data is just the results of our experiments and our observations. But what is that inference part? Right. That's the crucial bit. I've traded one word you thought you understood with another word that maybe you don't even think you understood inference. So now I need to help you define that. So inference is just taking, combining some data, some information we have that seems to be about the world with some model of the world. So we might call that a theory, we might call that a hypothesis, we might just call it a model. But it's some idea for how the world works. And usually that idea for how the world works has something that we don't fully specify. Right. You know, even if you take the idea that the sun is at the center and the planets are going around it, you don't know exactly where the planets are. Right. You have to, that has to be something that you measure inside the model and just say you need, you need to make that inference. And that's what science helps us do. It helps us both within a model, take information about the world and see how it fits into the model. Well, but it also, if we can, we can eventually compare the, the heliocentric model like that with the, with the old geocentric model with the Earth at the center, and we could see which fits the data better and which model we prefer, given how well they both fit the data.
Greg McNiff
I want to build on that, particularly the role of induction, or I should say the problem of induction as labeled by John Stuart Mill. I'll quote you quoting him or paraphrasing him. This is the problem of induction. Does reason alone allow me to generalize from some small or at least finite set of observed facts to the more general statements about the world? Andrew, just from the way you've described the scientific method, that seems to be how we proceed. Is that fair?
Andrew Jaffe
That. Yeah, this is it. So induction is almost all of the scientific method. At some level, we are, we are making observations and we're trying to combine them together and generalize from them. So we're trying to take the things that we've seen and figure out about things that we haven't seen.
Greg McNiff
Right.
Andrew Jaffe
That's all induction is. But that's kind of an amazing process. And what exercised a lot of philosophers from at least the time of David Hume, the Scottish Enlightenment philosopher, till today, and still is, is the worry that you can't prove that induction works. Right. We had, you know, people had this idea that the way science, broadly speaking, even before science was a word, natural philosophy, as it used to be called. Right. Was that we sort of had thoughts about the world and we proved them. We saw that they had to be true. And this goes back to, you know, when you learn geometry at school and you do proofs of geometry or proofs in mathematic and proofs in logic, but the world is not quite so tidy, right? And the world, you know, is a finite set of facts. And we want to somehow make a generalization about everything else. And it bothered people and to some extent still bothers some people that we can't do that. And the. And it's true, we can't do that. So we, you know, there really is no way to prove that having seen, having seen the sun rise for the last million days as a species, or, you know, way more than that, as a species, that we should. That we should. We're going to see it again tomorrow. Yeah, you can't. You need a model to prove that. You need the fact that the Earth goes around the sun. And we understand why that is. And with that model, we can say with near certainty that that will continue tomorrow.
Greg McNiff
Andrew, you write, induction makes sense if and only if the world makes sense. This is the idea of naturalism. And I want to return to that when we talk about quantum mechanics, naturalism versus determinism. But for purposes of this question, the overarching model that the world is at its bottom, intelligible, this is, I believe, the naturalism argument. Do you believe the world at its bottom, in essence, is intelligible?
Andrew Jaffe
Sure. And everybody else does too, in some sense, right? It's the only way you can proceed. If you believed that the world was unintelligible, that you couldn't apply your intellect to it and understand it, then you know, literally all bets are off. You, you then, you therefore believe that anything could happen. And then, you know, my book would be really, really well titled in that case, because the universe really just would be random in an even more profound sense than it really is. Like anything could happen in the next instance. And maybe that's true. Maybe we've been really lucky and things have been very regular for the last 14 billion years and, and tomorrow things are going to change randomly. And that's. That is logically possible. Right? That's. That was what bothered Hume. It is still possible. You couldn't, you can't prove that that's not going to happen. And he was right. You can't prove that that's not going to happen. And you need this, this overarching model, and it's a model too, that the world is intelligible, this naturalism argument, in order to make any sense of the world. And the alternative is so chaotic that at some level it's not worth questioning except within the models that we have. Is there a reason why, you know, things are regular even given that there are laws of physics? Right. So there's some regularity, but does it have to be as regular as we see? That is something you can test. It's impossible to test that there's, you know, the idea that there is regularity at all. I think we just have to take that almost as a given.
Greg McNiff
Got it. So I want to hit on that distinction. Is there a distinction between the world functions under certain laws and maybe it's mathematics. I know you quote Wigner talking about mathematics, so I want to ask you a. Is math the language of the universe and is it intelligible? And then to what you just said, is there a reason for why it's intelligible, why it's order? I mean, you talk about evolution as in we just know. You know, I'm not going to get this right, but, for example, entropy overall is a disordered state. But you can make the argument Earth actually benefited from a certain order out of entropy that we were able to evolve. And I think you say, like, we knew how to evolve. I'm getting this wrong. Single cells knew how to evolve. So my question is, is there an order? Is and is mathematics part of that order? And is there also a reason for why there's an order? How would you.
Andrew Jaffe
Yeah, so I think there is an order. Yeah, I think that is past all the tests that we've given it right so far. It is mathematical, but I think that's because that's our language. I don't think that's. I don't think the universe knows mathematics. I think other scientists might disagree, but I think the reason why mathematics works is because we have. There is regularity in the world. And we invented for lack of a better term. And again, people could disagree about this. The mathematics, we need to describe it. And so it shouldn't be a coincidence that this, that these pattern matching rules that you learn when you're, you know, doing arithmetic and calculus or whatever, not to mention group theory or category theory or something like that in higher mathematics. These, we have invented these rules to be useful and it shouldn't surprise us that they are.
Greg McNiff
Makes sense. If I could push back slightly and you quote Wigner saying the mathematical symbol PI appears in the formula for the bell shaped Gaussian curve. That, that seems amazing. And I've heard discussions around fractals show up in different places. How would you explain that? Seemingly there's like a, maybe you could almost argue a universal mathematical language or somehow two unrelated events or dynamics seem to have some mathematical correlation. Is that fair? Is that a fluke? How do you think about that?
Andrew Jaffe
Well, it is certainly true, right? PI appears in lots of places. We think the definition of PI is something to do with the radius and circumference of a circle. But we could equally well take the definition of PI to be this number that appears in the formula for the bell shaped curve. And we could wonder why is, why does that, why does that bell shaped curve number appear in the circle? So you know, there's no one that's more fundamental than the other. And you know, you can, you can. For that case, there's actually, I think it's actually pretty easy to write down a geometrical statement about the way the integral of the bell shaped curve, if you really made a two dimensional bell shaped curve, there's a circle there now and you can actually kind of understand why things having to do with circles would appear in both places. So it's, so that's a case where it's not totally magical. PI appears in lots of other cases that might seem more magical. But I think if you, if you trace things back carefully enough then, then there's a, there's a reason why mathematics holds together the way it is, right? This is, this goes back to stuff that, you know, the logicians of the early 20th century tried to show how from a certain set of axioms you could prove all of mathematics. Now it turns out that, that, that's even more complicated than we think because of people like Godel and, and his proofs about how actually mathematics isn't complete in some sense. But I think the kinds of things that we're talking about that doesn't really come in, it's just the fact that there are nonetheless within our system of mathematics, things that are related to one another in subtle and interesting ways that can teach us things. But I don't think it's magic.
Greg McNiff
That's interesting because when we get to quantum mechanics, you talk about the Copenhagen theory, the many worlds theory and Cubism, which means quantum Bayesian. And I'm jumping ahead, so I'll circle back to our discussion here. But even there you suggest Q bism is more probabilities as judgment as our understanding as opposed to an ontological explanation for the, for the world. And it almost seems like you're suggesting mathematics helps us understand, but it's not part of the underlying ontology or structure of the universe.
Andrew Jaffe
Yeah, I think, I think the universe is, and I can't. Some 20th century philosopher used this terminology. Unfortunately, I don't remember who. But the universe is just one damn thing after another. And, you know, and that's all it is. It's just the stuff that happens. And yes, with some regularities, but it's, you know, we are the ones who choose to describe that by differential equations and integrals and things like that. And yes, it works. It's not a. I'm not saying that we are imposing our mathematics on the world in that sense. It really is a good description, but it's a good description because, you know, once you have things changing with time, then derivatives come in.
Greg McNiff
Yep.
Andrew Jaffe
Right. And once, you know, once you have things accumulating with time, integrals come in and, you know, and, and, and things like that. Right. So it's, it's mysterious in detail because, you know, the, the mysteries are, you know, why do these simple laws work, not why do laws work? I think, yeah, you know, we've been lucky that, that the laws of physics that we've discovered have been straightforward. I don't know if that's a particularly good term. And that they've kind of, when they've been superseded, like when Einstein superseded Newton. Newton's theory is really beautiful and it's very simple. Newton's theory of gravity is just essentially one formula and his other, and his other laws are just a few other formulae. And, and Einstein's version, which superseded that is also beautiful in a different way.
Marshall Po
Yeah.
Andrew Jaffe
And we shouldn't be fooled necessarily into worrying, into thinking that the beauty part is one of the laws of the universe. I think some scientists do. And you can therefore go down rabbit holes where you, where you pursue a beautiful theory that just isn't right.
Greg McNiff
Yeah.
Andrew Jaffe
And maybe, you know, I'm not going to cast aspersions on some colleagues. I know that there are, you know, that. That some people think that different. Different efforts in physics today are beauty at the expense of reality. And maybe that's true. But more to the point, people have done this forever. You know, it's. It's. Maybe things like string theory get a lot of press because they seem like, oh, it's a lot of effort on this one thing and maybe it's not right because it seems so beautiful. But this isn't new and it's not. And they're not being foolish. They might be wrong.
Greg McNiff
Yeah.
Andrew Jaffe
But, you know, there's. There were also hints that there's reasons why it is correct. And so that's why they pursued it. It wasn't that it was obviously incorrect, but they pursued it at the expense of, you know, at the expense of the data because they wanted. They thought it was beautiful. They pursued it because there are intriguing hints in the structure of that theory about the way the world seems to work now. And so that the simple version that we observe, just like Newton's simple version, contains the seeds of Einstein. It looked like the simple version that we observe contains the seeds of some of these grander theories. But it might be that none of them we've thought about so far are the right grander theory. We just don't know yet.
Ad Voice 2
Meet the computer you can talk to with Copilot on Windows. Working, creating and collaborating is as easy as talking. Got writer's block? Share your screen with Copilot Vision to help spark inspiration and use Copilot voice to have a conversation and brainstorm ideas. Or maybe you need some tech help with Copilot Vision. Copilot sees what you see. Let Copilot talk you through step by step guidance so you can master new.
Ad Voice 1
Apps, games and skills faster.
Ad Voice 2
Try now@windows.com copilot hi, I'm here to.
Andrew Jaffe
Pick up my son, Milo. There's no Milo here.
Ad Voice 1
Who picked up my son from school?
Greg McNiff
Streaming only on Peacock.
Andrew Jaffe
I'm gonna need the name of everyone that could have a connection. You don't understand.
Ad Voice 1
It was just the five of us. So this was all planned? What are you gonna do?
Andrew Jaffe
I will do whatever it takes to.
Ad Voice 1
Get my son back.
Andrew Jaffe
I honestly didn't see this coming. These nice people killing each other. All her fault. A new series streaming now only on Peacock. Extra value meals are back. That means 10 tender juicy McNuggets and medium fries and a drink are just $8 only at McDonald's for a limited time only. Prices and participation may vary. Prices may be Higher in Hawaii, Alaska and California and for delivery.
Greg McNiff
Let me ask you about that. Clearly, it seems like the Holy Grail is some sort of final unified theory, quantized gravity, or merging quantum mechanics and the general theory of relativity. Do you think there is one formula, Andrew, to, to sort of unite them all? I mean, do you think, do you think.
Andrew Jaffe
I suspect that there is a theory that makes sense. It might not be quite uniting them all in the way we think. Yeah, it has to be different than what we have today because what we have today simply doesn't make sense at the highest energies on the smallest length scales, things like that. So it just, so it's. So we know that our theory isn't complete, but we don't know how it's completed. And it might be completed in a very messy way. And maybe gravity ends up, yes, being fundamentally different from all the other forces, but maybe it doesn't. We really don't know yet. I would love to be able to find that out during my lifetime. And maybe we will. And that is, you know, that is a, that is an empirical question that, you know, my, me and my colleagues will, will either know the answer to or not eventually.
Greg McNiff
Absolutely. You end the chapter with. On Induction by saying what we want is to quantify induction. And this introduces a real theme of the book, probability. A. I'm going to ask you, could you talk a little bit about the role of probability and particularly the principle of indifference?
Andrew Jaffe
Sure. So probability is. Is what you think it is. Right. Something is. If something has a higher probability, it is more probable. That is, you know, you think it happened. You are more, you are more likely to think that you think it happened. More likely. And we give probabilities between 0 and 1. But that's a convention. We could have done it in lots of other ways. We say if something has probability zero, it definitely didn't happen. And if something has probability one, we say that it definitely did happen. And we can, we will. I know that we're going to talk in a few minutes about what we mean by the definitely is there. So I'll leave, I'll leave that hanging just for a little bit. And the, and then the question is, how do we deal with everything in between? And the simplest version is, is a flipped coin. Right. So, you know, it's everyone's favorite example for probability. I flip a coin, it's heads or tails. And I, you know, I ask you to, to tell me what's the probability that it's heads and. Well, I'll ask you what is the Probability I flipped a coin, it's heads or tails, what's the probability that it's heads? Yeah.
Greg McNiff
I assume you're a fair man. So 50% or 1%.
Andrew Jaffe
Exactly, yes. And you know what, even, even if I wasn't a fair man, unless you knew that I was betting against you, in which case I had some reason to care, then, even then you'd probably have to say 5050 because you have no other information that gives you anything else, right? If you knew that I wanted you to lose a bet, then I might, you know that then I might have done something more clever. But if I'm just asking you whether it's heads or tails, even if, even if you thought maybe I had a two headed coin or even a two tailed coin, if you didn't have any information about what I might actually have in my pocket, then you're just going to say 50 50. So that's the principle of indifference. When you have no reason to prefer one selection over the other. And I don't mean prefer as in want to happen, I mean prefer as in think it did happen, then you choose between those and you can expand that to, you know, the 52 cards in a deck of cards. So what is the probability that you're going to pick the ace of diamonds out of the deck is 1 in 52 as long as there are no jokers in the deck, right? What is the probability that the roulette wheel will come up? And then I forget how many slots are on a roulette wheel, but it's whatever one in that number, right? So that gets you pretty far in kind of calibrating what probability means when you have these indifferences. Because of course you can bunch them together as well. You can say if you have a deck of cards, what is the probability that the card is a diamond, right? There's four, there's four suits, there's the same number of cards. So, so then it's 1 in 4, even though it's really 13 and 52, right? But 13 and 52 is the same as 1 and 4, so you can use that. So you can kind of bootstrap your way up to lots of different probabilities for cases where that, that, that is a good description. But we can use probabilities in lots of other cases too. And then we might have to use the mathematical rules of probability. So a lot of the formulae, as you mentioned before that I put in the book, are the probability rules, because they're kind of simple, they don't involve a Lot of complicated mathematics, but they really do describe the way we intuitively use probability all the time to figure out how likely things are when we are guiding ourselves through the world, even if we don't always. In fact, we almost never do it with the mathematical precision that a true probability would have.
Greg McNiff
But, but, Andrew, like you just said, every day we're calculating probabilities. You get the example of the boss, the weather, you know, I don't know, casino. Obviously we can calculate them. Right. I mean, we're doing that intuitively. Our lives are almost lived in a probabilistic fashion, so to speak.
Andrew Jaffe
Yeah, because it's a random universe out there. And so the only way we can tame that universe, tame that randomness, is by assigning probabilities.
Greg McNiff
Perfect. I'm going to hit you with the Italian statistician Bruno Finetti's quote. Probability does not exist. And I should say this is under radical subjectivity. Does this just get back to probability is a mental construct we use. How. How would you say? How would.
Andrew Jaffe
Yeah, so, so that's exactly right. So what he meant by it is that it's not a thing that's out in the world. It's not a, you know, a fundamental particle has a mass, it has a charge, it, you know, has angular momentum. There are all these numbers you can attach to it. You can't attach a probability to it. There's no fundamental probability in the world. It's all about the models that we make of the world. Now. They might be models upon which essentially everybody agrees.
Marshall Po
Yeah.
Andrew Jaffe
So we all assign the same probabilities without a lot of controversy. But they might be models in which we have, which people can disagree when it comes to, you know, the efficacy of, of policies, political policies. Right. You know, we all, when we're putting things together, if we have to do things like that, we say, okay, I'm pretty sure this is going to have the effect of lowering poverty or whatever in the world. And my, you know, if, if I think that this particular action will, will do a good job, I have a high. I assign a large probability to it. You know, you may be of a different political persuasion and assign a small probability to it. So those are probabilities that are clearly not in the world. They're in our heads. And the only way to, to, you know, we. And we use them differently. And then, of course, we can do experiments and see who might be more right or wrong. But that comes after the fact. That's not something that's inherent in the world. It's inherent in our understanding of the world.
Greg McNiff
Absolutely. If there was a key theme running through this book, I think it would be. Or one of the top themes would be all probabilities are conditional. What do you mean by that?
Andrew Jaffe
Yeah, so that's my version. So you know, Dfinity slogan people can, you know, especially when we get to quantum mechanics, we'll, we'll talk about whether there are probabilities that are in the world. But Definetti said that they don't exist. But my version is that all probabilities are conditional. That being conditional means that you don't just say well the probability of heads is a half. You say the probability of heads is a half. Given my model for you're flipping the coin. So given that I think, you know, it could either have been heads or tails and I don't have any reason to to pick which, then it's one half. Right. If, if the model is instead aha. I know that you, you just flipped a two headed coin then the probability is one. Right. It just depends. Depends on the model and the same world. Right. I could have flipped a two headed coin, had it in my hand. And if I don't tell you that the, that it's a two headed coin then you're going to do 50 50. And once I tell you, you that it is, then it's changed from 5050 to 100% and that's because it's conditional on the model you have in your head for what's going on.
Greg McNiff
Got seems like there are two schools of probability and that's probably a simplification. But the frequentist and then the Bayesian school and, and Bayes is clearly a hero or a very important individual in your book. And candidly, when I was exposed to a business school I thought it was fascinating his background and the way he approached this and frankly, unfortunately, he didn't live long enough to see the impact. Side note, I love the fact the hedge fund in New Jersey is paying for the upkeep of his grave in a. There are some very famous people in that cemetery. I'm blanking on them.
Andrew Jaffe
But maybe Blake, William Blake is the famous poet, he's buried there.
Greg McNiff
But Andrew, could you briefly describe frequentists and then how Bayesian differs from frequentists and I think why you think a Bayesian approach to life or at least some of these big questions in science is the right approach?
Andrew Jaffe
Sure. And I'll start by saying that in some sense they're both correct. There's nothing wrong with. They're mathematically consistent ways of looking at probability. So a Bayesian is what we've been talking about. It's you as, and I guess I've been using the word, we haven't really delved into it, but you assign a probability. So it's just I, you know, I, I make my model of the world and then conditional on that model of the world, I assign probabilities. And I could, I could hold two different models in my head at the same time, right. I'm vast, I contain multitudes, as Walt Whitman said. And so I could just, you know, they could be as if probabilities. If I choose this model, I believe this. If I choose this model, I believe, I believe these other probabilities. Frequentists want probabilities to be a little more about the things that are in the world. And so for a frequentist, a probability is defined to be a, a fraction, right? So if, if I flip a coin a million times and it comes up heads, you know, 532,248 times, then I can test the hypothesis that it's a fair coin that it was going to, you know, that it really was going to be 50, 50 each time, depending on that number. Right. And what the frequentist would assume is that if I repeated this experiment an infinite number of times, then that fraction would converge to 50, 50, right? Now, one problem there is the bit about repeating an experiment an infinite number of times. No one has ever done that. No one will ever do that because there's not an infinite amount of time to do an infinite number of experiments. So you can never. So even for a, for a frequentist, the probability doesn't really exist except in some notional experiment that you could imagine having happen in your head. But again, we're then getting into things happening in heads more than hat, more than things happening in the real world where the, where the real difference comes between what a frequentist thinks about probability and a Bayesian thinks about probability is how it's used. So a Bayesian is very happy for the outcome of an experiment to be one of these bell shaped curves that we talked about before, where you say, okay, I am, and there's a reason why I picked this number. 68% certain that, that, that the value of the expansion rate of the universe, I'm just picking a nice cosmological example, is between about 66 and 68 km per second per megaparsec. That's the weird units that we measure these things in. Yeah, and, and I, and what I mean by that is the kind of probability I was talking about before that, you know, I am, that's how sure I am. Right. And, and you can, going back to gambling, you can usually relate these things to betting odds. So if I really, really believe that that's the, that that's the probability, then then those are related to the odds. I would take on a bet about the actual outcome of some future experiment to really measure this thing even better. Right. So that's what I mean by a Bayesian probability in a sense like this. And a frequentist doesn't let us ascribe probabilities to truth to things that might be true or false in the world. They only ascribe probabilities to these long run frequencies. So you have to sort of play a little gymnastic game in your head where you say, okay, imagine I repeated this experiment in some notional way to measure this expansion rate of the universe a large number of times or an infinite number of times. What fraction would, you know, would be within 68% of some number that I choose as my estimate of that number? And it's a much more convoluted thing that in a convoluted, I'm a physicist, we do lots of convoluted things. That's, that's not really the problem. The, the problem is that it's not really measuring the, measuring the number that I want to measure. Yeah. So that's why the Bayesian thing makes sense now. They should, you know, in the long run, if your error bar gets really, really small, then it's saying that, yes, there are reasons why you're warranted to assume that it's really, really close to this number. But it doesn't really, it's, it's much more difficult to interpret sort of a wide error bar in this case. And in cosmology, there's lots of things we're never going to measure really well. And so we need to be able to think about error bars a bit differently. So at the end of the day, I'm for sure a really thoroughgoing Bayesian. But I will say that there are things that, that a frequentist school of thought is good at assessing things like, you know, how if I do do an experiment a bunch of times, you know, how can I, can I use that idea to test how good an experiment it is, how severe a test of my theory that it is, and some of my colleagues who have sort of taken a, a particular form of frequentism called error statistics really lean into this idea that it's, that what it's good at is Assessing the. How, how high a quality your experiment is relative to what you want to measure with it. And I think it does have some use in that case. But you still have to be very careful when you're reporting things like these famous or infamous P values that people who are a little bit familiar, I mean it does make it into the, you know, the newspapers, these ideas about, you know, how, like how effective drugs are and things like that. They're often quoted with these numbers called P, which sort of stands for probability, but not in a Bayesian sense. And it, it's, it's about how, you know, how in a long run of experiments how often you would get a outcome different in a particular way from the one that you got. And it is again very convoluted. It's hard to describe, but it's very hard to interpret those things.
Greg McNiff
No, you do a nice non quantitative description. When you quote John Maynard Keynes, he says when the facts change, I change my opinion. Sort of a nice synopsis of Bayesian. And if you for example, were explaining it to a group of non, I guess, non stem, non stats, how in layman's terms would you say Bayesian is it. It allows us to update our assessment or our probabilities based upon new information.
Andrew Jaffe
Yeah, it's a model. Everything's a model. It's a model for learning.
Greg McNiff
Yeah.
Andrew Jaffe
You know, you, you start out with some, and we, we haven't used any of the magic words of Bayes's theorem yet, but you start out with some prior information. Yeah, right. And that could be real prior. It could be before you've, you know, all you have is your model. You do need this model. But if all you have is the model, that's your prior information. So maybe there are some numbers associated with your model that you'd like to measure, like the expansion of the universe that I mentioned before or the, you know, the efficacy of some drug or something like that. Right. You have some model and you want to measure this number based on some data. And so you do an experiment that gives, that transforms your prior into your posterior. Right. And these are kind of time based phrases, prior and posterior. But they don't really have to be prior in time and posterior in time. They could be, you know, you could have your data and then do all this. You start with your prior, it transforms it into your posterior. If you then get new data about the same question, that old posterior can become your prior for your new experiment. Yeah. And so on and so on and so on. So that in that sense It's a model for learning. We convert all of our old priors, combined with all of the data that we've gotten into our heads so far into our current prior for the universe or the, the posture we have now when we encounter everything else in the world becomes our prior gets transformed tomorrow into tomorrow's us, today's posterior and tomorrow's prior.
Greg McNiff
Yeah, it almost seems recursive or iterative or. Yeah, keeps. I mean, it actually seems sort of the way. I'm the last person be saying this, that these new deep learning neural net models function as they keep updating and correcting and taking in new information. But that's probably a separate discussion. So you laid this out wonderfully. So I'm going to hit you. I'm going to tie this back to Hume here. How is Bayesian probability answer to Hume's problem or issue with induction?
Andrew Jaffe
Well, so it doesn't solve it so much as dissolve it.
Greg McNiff
Right.
Andrew Jaffe
Um, so, you know, there is no solution. He was right. You cannot logically prove induction. It's. It's just as simple as that. You know, he, he put it in a very different context having to do with morality, but it's kind of the same idea. You can't derive ought from is.
Greg McNiff
Yeah.
Andrew Jaffe
You know, you can't derive what we ought to think about the world from the way the world actually is. You have to, you have to put other information in. And so that's, and that's how Bayes solves Hume's problem of induction by putting in a model. And that model, hopefully can be pretty bare bones. Then we can learn about the universe. And in fact, it was really done well by Bayes's friend and sort of successor, the person who read his paper posthumously to the Royal Society, this guy called Richard Price, who, Who invented actuary science and did a lot of other things. He was apparently a good friend of Thomas Paine, who wrote Common Sense at the time of the American Revolution. And he really did apply Bayes's theorem of, you know, avant la lettre, as they say before it was called that even, you know, Price didn't call it that. To, to the problem of just, you know, if the, if the sun has risen every day, what is the chance it's going to rise again tomorrow? If you have no other model other than that and that, you know, there is just, there's some probability that the world is going to be the same tomorrow as it was today, essentially. And if that's your only model, that there's just this number that describes that every day. Then you can you can work out exactly what that probability is for any number of, you know, successful times when you've seen it happen in the past. And Pierre Simone Laplace did this in great detail in the in the 1800s after, after Bayes and Price. But they were the ones who really first thought about this problem in a probabilistic way. So the way it solves it or dissolves it is once you have a model, you can become arbitrarily sure of things if you keep observing them in the world.
Ad Voice 3
This episode is brought to you by Amazon Prime Black Friday Game day on prime is an epic day of live sports. It all starts at 9am Eastern with the Capital One skins game. Then Black Friday football returns when the Bears take on the Eagles at 3pm and it culminates with the final night of Emirates NBA Cub group play with Bucs Knicks at 7pm and Mavs Lakers at 10pm Black Friday game day only on Prime.
Ad Voice 1
Ford Blue Cruise Hands Free highway driving takes the work out of being behind the wheel, allowing you to rely on, relax and reconnect while also staying in control. Enjoy the drive in BlueCruise enabled vehicles like the F150 Explorer and Mustang Mach E. Available feature on equipped vehicles. Terms apply. Does not replace safe driving. See Ford.com BlueCruise for more details.
Ad Voice 3
More rewards, more savings with American Express Business Gold. Earn up to $395 back in annual statement credits on eligible purchases at select Shipping, food delivery and retail subscription merchants. Enjoy the benefits of membership with the Amex Business Gold Card. Terms apply. Learn more@american express.com Business Gold AmEx Business Gold Card built for business by American Express.
Greg McNiff
And I know you note someone calls Price actually the first Bayesian. I didn't realize what a phenomenal mathematician he was. He always gets it seems like he's a footnote to history, the individual who brought Bayes to the public awareness. But in his own right. He was a very strong mathematician, no doubt.
Andrew Jaffe
No, he was. It turns out he's a very. I hadn't really heard of him either, I admit, but he was actually very well known at the time and yeah, in certain circles anyway is still very well known.
Greg McNiff
Yeah, no, I'm sure. Before we move on, I think you talked about in the beginning writing about music and non mathematical topics. Blake actually figures obviously in the same cemetery. But also in your book here, do you think Blake's a Bayesian? What's your view of Blake and subjective?
Andrew Jaffe
Yeah, well, yeah, I think Blake would have been if Blake was a Bayesian. He was a very subjective one. So, you know, Blake, Blake was a sort of romantic poet, you know, in the sense of the Romantic movement. But very early on he wrote all this visionary poetry and, and prints that he made and paintings and he created this, this miraculous and magical cosmology, for lack of a better word. Very, very religious. So he was a very religious, but not, not ascribing to any particular sect, I think would be a fair, fair statement of Christ. He was a Christian, but, but beyond that, I think he was his own sort of Christian. And so he really didn't like the idea that he had to obey the, well, the, the human laws of any sort, except for what he thought was correct. And he conceived that Newton's laws. And he was. This was quite a bit after Newton. Yeah, that Newton's laws were that sort of law. He really didn't see a difference between laws of physics and human laws about how you should act. And he had a famous quote that I don't have in front of me now about how, you know, he, he, he, he will not basically obey other laws. He will be. His business is to create. So he wanted to create his own laws of the world and obey them. And I think if he had to choose between, between a, a sort of arid physics which everything is just prescribed and we already know everything and, and one in which at least to learn you have to put in your own model and, and there's a subjectivity there at the very, at the very bottom. I think he would certainly prefer that. Whether he'd be an actual Bayesian probably. He wouldn't really like the mathematics, but.
Greg McNiff
No. Good job on getting him in here, though. Fascinating figure. We need to move on here. You described Darwin's theory of evolution as an algorithm. Why is that?
Andrew Jaffe
Well, so an algorithm is sort of a series of steps that, that you, that you, that you put some data through or something through to get to go from something to another. And that's what, that's what evolution is. If you start with random mutations that can be passed on from, between, you know, from, from, from parents to offspring and there are pressures in the, in the universe that these, that the, that the creatures are living in of any sort of, then, then it is inevitable that, that random mutations will change the beings over time to be more suited for their environment. There's really no ifs, ands or buts about it. There's nothing, there's certainly nothing magical about it whatsoever. So given that essentially everybody agrees that those conditions do obtain right that there are random mutations, that there are pressures in the environment and that the random mutations are passed on from parent to offspring, then evolution does happen. I think there's no, there's no question about that. I guess there, if anybody questions evolution, it's whether that algorithm is all that is out there that is causing the difference in species. Most of us in the sciences would say of course, yes, but people of goodwill can disagree as they say, no, absolutely.
Greg McNiff
I know a niche construction or some sort of higher level evolution is certainly a topic about emerging properties and how things evolve in the right way. But I want to you have a really fascinating discussion and forgive me, I'm going to get Imre Lakatos name wrong but between him and Paul Firebend on this notion of what is the scientific method. And they seem a, to have different opinions but also to be friends. And could you briefly describe their views of how they view the scientific method?
Andrew Jaffe
Sure. So the Katos kind of took up the mantle of Carl Popper who's sort of a mid 20th century figure who kind of his, his insight, which is pretty good so far as it goes, is basically that you know, scientific theories are falsified. They're not, they're not, you know, they're not proven. So we, we agree, I totally agree with that part of it, but that they're, they can be falsified. So you can have an experiment that proves that a given theory can't possibly be correct and, and that that works to some extent. You have, there are a lot of caveats and I think the Hopper was sophisticated enough that he understood these caveats that you know, you, that because everything is probabilistic, you can never even really truly falsify a theory a hundred percent as it were, but nonetheless you could do a really good job. And furthermore that the real, the scientific method proceeds without just falsification. We do lots of actual measurement where we're trying to determine what things are not what they're not. And okay, you know, you could play a language game and just say, well by determining that the expansion rate of the universe is between 66 and 68, then I've proven that it's not outside that. Okay, but I'm not really sure that it's not outside that. And I need, I think something that he wasn't particularly keen on. I need probability to describe, you know, the bell shaped curve usually of how I think it's located between these two numbers and exactly why. So I think that is more of a sophisticated view of it than, than the simple view of, Of, Of Popper and Lucatos took that up where he didn't concentrate so much on individual theories, but he called scientific programs where it was a collection of things that a given set of researchers would be working on, essentially, and you kind of figure out what is the most crucial bit. So if you know you're trying to test general relativity, well, let's see, that also requires understanding black holes, but also in order to do that, you kind of also have to understand other stars in the universe and how they evolve. And you really also have to understand how your telescopes work. And sort of these. You know, that you sort of think of this as being these nested spheres, the. The core which has the real thing you really want to test the most at the center. And all of this, this. This not quite extraneous, but less. Less crucial stuff that you could imagine being not quite right, being not quite fitting with your base theory, but still not disproving the theory in the middle right. Disproving the core, the belt and the outside. Eventually you get to things that are much more like your theory about how your instrument works and not so much your theory about how black holes work, and you still need your theory about how your instrument works in order to prove the theory about how the black hole works. And he was good at sort of showing the way that these things are nested within one another and how that that set of things could then be put into sort of a larger program in which you could see how by continued experiment and continued theorizing, if you do it correctly, that you could build up more information about the core theory that you're trying to test general relativity or. Or indeed evolution or something like that. And that was in contra. And that was a sort of a progressive scientific program. And he contrasted that to, I think the term was degenerating scientific program, where you would. You would go back and you'd sort of make changes to the core as your data kept changing in order to make it still agree. And he. And again, people can disagree, but, you know, he put things like theories that he would call pseudo scientific, like Marxism and psychoanalysis. And again, I'm not gonna. I'm just using his examples here about, of. Of whether, you know, of. Of those things and called them degenerating theories because of the way their adherence would at least sometimes. And I think even their current adherents would agree that this sometimes happens. Would. Would be willing to accept almost any data as being, if not incompatible with their theory, sometimes even a proof of their theory. Right? Even though the Marxist revolution has not happened over the whole world after the Russian Revolution, nonetheless, that still seems to be able to be upheld, even if the sort of easy reading of Marx anyway says that that shouldn't have happened. Right. Things like that. And of course, the worry of any of us, and I think Licatos diagnosis is correct. You know, the worry of any of us doing science is that that's, you know, we could be working on a theory that turns into that, or we. Or we turn a theory into that by what happens. And, you know, we can skip ahead, you know, later on, you know, talk about the current cosmological theories. And indeed, we are worried about some tensions within those theories. And we, we sometimes do add on bells and whistles to accommodate the data, and they are not things that were in our theory to begin with. Yeah. And there is a worry. I don't think it's a strong worry necessarily, but of course I'm embedded in it, so people can disagree. But it's a. There's a worry that some of these dark things that exist in our cosmology that we observe the effects of but can't, you know, but don't understand, might be something that could turn our theories degenerating over time, even if they haven't yet. Yeah.
Greg McNiff
Okay, I want to move on to part two, which you label the random universe. And in the first two chapters, you talk about entropy. And I want to. I'm going to hit you with another quote and ask you to explain, but you basically conclude. But to really understand entropy from Clausius's point of view and Boltzmann's and Shannon's, and you've been talking about entropy and information theory, we need to return to our slogan all probabilities are conditional. How does Shannon's information theory, Clausius notion of entropy, and Boltzmann statistical mechanics all rely on these conditional probabilities.
Andrew Jaffe
Sure. So entropy is. Many people are familiar with entropy and they sort of just equate it with randomness of the universe disorder. And I think a good way to think about it. So we all learn about. Well, no, we. Those of us who've taken a physics class and, you know, learn thermodynamics. You learn about entropy in the context of the second law of thermodynamics, which says basically that on the whole, entropy increases, things get more random over time. And, and entropy was originally invented before we knew about molecules or atoms. It was invented to describe energy that became inaccessible to. To use for engines, essentially. Right. And because this was really at the time when people were, that thermodynamics was being used to describe the coming industrial revolution and the things people were beginning to build and entropy was stuff that you could no longer make good use of to. And so that, and that's because it basically is indeed about we, that we eventually learned about the randomness of the particles that make up the substances we're talking about. And because we lack information about the exact trajectories of those particles, we can't use them to move things around, can use them to expand a whole balloon of hot air. But that's, that's not quite the same thing as extracting really useful work on a microscopic scale. But that's because we don't have the information. If we could, and Maxwell invented a demon in his head that, that could do this, if we somehow knew the individual trajectories of particles on the very smallest scales, we could imagine separating out the hot, the, the fast moving atoms from the slow moving atoms and separate the, the hot gas from the cold gas and do something that never happened spontaneously, right? Because this would be entropy decreasing rather, rather, rather a lot. If you, if half of your, if you had a box of, you know, of 70 degrees Fahrenheit gas and you ended up instead with a box that was 100 degrees on one side and 40 degrees on the other, you know, with a real sharp dividing line in between. That never happened spontaneously. But you can imagine a creature, a tiny creature being able to do that somehow. And that's because that creature has this information, right? It never happened spontaneously. So finally getting to answer your question, that's because the, the, the really entropy is about probabilities. The formula that Boltzmann and Shannon used to define entropy, so not the one having to do with substances and gas, but having to do with probabilities, was indeed about probabilities and what they wrote down. And also Josiah Willard Gibbs in the US in the, in the 1800s wrote down formulae that have to do with the probabilities of things. And if, you know, if you have a better probabilistic model for the individual particles, then you can do things that seem to disobey these laws about entropy because they're laws about conditional probability. It's just that in most cases we don't have conditional information. But for Shannon that actually was really obvious. And because for what he did, he, he had to, he applied entropy to information and you then really have to work out what the different models are for the kinds of messages that you're going to send he wrote these laws down about communication. So the kinds of messages that you want to send through a device, the ways in which those messages might be degraded in your device, and those are very conditional probabilities because, you know, the, it's all about exactly the probability that a bit in, you know, that you're sending down a wire will be converted from 1 to 0 or 0 to 1. And without a pretty detailed model for that, you can't use Shannon's formula at all. And you can't use any of the rest of the infrastructure that he really amazingly developed in the 40s and 50s.
Greg McNiff
Definitely, Shannon was amazing. Boltzmann's fascinating. Why do you call his equation so remarkable?
Andrew Jaffe
Oh, because it's. First of all, it was, you know, really came out of his head. He had no detailed knowledge of the microscopic world. They didn't know quantum mechanics, so they really couldn't. They were beginning to have an idea that there were things like atoms and molecules, but they had no real understanding of their behavior. But he was able to write down an equation which married this large scale thing that was invented for engines to a microscopic but unobserved theory about the, the way gas is made. And it was mostly about gases, but also it applies to solids and liquids, but really largely about gases. And, you know, and it was, and it's so beautiful and simple. It's just S equals K logarithm of Omega and S means entropy. K is just some number which we've named after Boltzmann. In fact, he didn't name it after himself, but we named it after him. Log is the logarithm, which, you know, you may remember from your math class. And Omega, in this very simple version of the probabilistic model is just the number of possible states of your system. And that really means the number of ways that all of those different atoms could be arranged. And so that number, when you're talking about something with, with let's say, you know, a cubic met, you know, a cubic yard for Americans of, of gas has about one followed by 23 zeros of particles in it. That's a lot of particles. And so the number of possible states is even larger than that. Right? They could be. They could all be moving around in lots of ways. So it's a vast number. And but he was able to show that this vast number was exactly the right way to get these much earlier definition, to get the same number as these much earlier definitions of this entropy that had to do with, with, you know, work and energy. It's that was really a miraculous leap of insight that that we still use all the time today.
Ad Voice 2
This episode is brought to you by State Farm. Listening to this podcast. Smart move. Being financially savvy. Smart move. Another smart move. Having State Farm help you create a competitive price when you choose to bundle home and auto bundling. Just another way to save with a personal price plan. Like a good neighbor, State Farm is there. Prices are based on rating plans that vary by state. Coverage options are selected by the customer. Availability, amount of discounts, and savings and eligibility vary by state.
Ad Voice 1
This is a real good story about Bronx and his dad, Ryan. Real United Airlines customers.
Ad Voice 3
We were returning home and one of the flight attendants asked Bronx if he wanted to see the flight deck and meet Kath and Andrew.
Andrew Jaffe
I got to sit in the driver's seat. I got. I grew up in an aviation family, and seeing Bronx kind of reminded me of myself when I was that age.
Ad Voice 1
That's Andrew, a real United pilot.
Andrew Jaffe
These small interactions can shape a kid's future. It felt like I was the captain.
Ad Voice 3
Allowing my son to see the flight deck will stick with us forever.
Andrew Jaffe
That's how good leads the way. Shopping is hard, right? But I found a better way. Stitch fix online Personal styling makes it easy. I just give my stylist my size, style, and budget preferences. I order boxes when I want and how I want. No subscription required. And he sends just for me pieces, plus outfit recommendations and styling tips. I keep what works and send back the rest. It's so easy make style easy. Get started today@stitchfix.com Spotify. That's stitchfix.com Spotify.
Greg McNiff
Would you say it's a product of a Boltzmann brain?
Andrew Jaffe
Well, a Boltzmann brain. So it was literally true? Yes, it was a product of. Of the Boltzmann brain. If you want me to tell what this sort of hypothetical idea of Boltzmann brains, maybe we'll get to it at the end.
Greg McNiff
But yeah, that's crazy. And moving from crazy to completely random or uncertain. I want to move to quantum mechanics. And you spend a fair amount of time in the book talking about the Copenhagen infiltration. And I'll ask you to describe that in the many worlds. Maybe if you could briefly set it up. And then I want to ask you, you specifically say the many worlds theories is, quote, theoretically parsimonious but ontologically extravagant. I have to ask you to unpack that phrase, but sure. Could you just set it up for us first?
Andrew Jaffe
Okay, so what is quantum mechanics? Quantum mechanics is our theory of almost everything that we Observe. But it mostly matters on very small scales. And, you know, and it was first invented to describe in particular, the. The way that. That. That atoms behave, the way they have energy levels. So we all know, we all have this picture in our head that an atom is like a mini solar system with a. With a nucleus, protons and neutrons in the center, and then electrons orbiting around, just like in the solar system, except that those orbits are not really orbits in the same way at all that are. That our planets orbit the sun, even though some of the physics is very similar, because it's on the small scale, because the laws of quantum mechanics hold, the orbits are much more about probabilities. So in quantum mechanics, everything comes down to a probability. So the thing that we can calculate in quantum mechanics, those things are really all about probabilities. So how probable it is to find an electron at a particular place, how probable it is to find an electron with a particular energy.
Greg McNiff
Right?
Andrew Jaffe
And it's called quantum mechanics because around an atom, the possible energies are a discrete set. So, you know, the, the. And. And people wrote down formulae that described the allowed values of energy. And before the quantum mechanics that we know today was invented, people had seen that they had this discrete set and that. And. And using, you know, Latin words, that was called quantized. So rather than continuous, they're quantized. And so the. The theory of the mechanics of that was called quantum mechanics eventually. And indeed, the miraculous or weird or still confusing thing about it is that the only thing quantum mechanics predicts are the probabilities that you will find certain things when you do experiments. And unlike the thermodynamics we were talking about before, where to some extent you could say, well, even though thermodynamics becomes about probabilities because of this Boltzmann stuff, it's not fundamentally about the probabilities. It's really about these particles. It's just that there are lots of them. And so we don't. We can't really ascribe individual details to them. But quantum mechanics seems to be really fundamentally about probabilities. And in order to come to grips with that, we've had to take these formulae and interpret them. We don't seem to think we have to interpret Newton's laws, they kind of. Or even Einstein's laws. You know, we do to some extent with Einstein, we have to think of this space time and these curved sheets that we like to think that everything happens on. But the interpretation of quantum mechanics seems more fraught because it's about probabilities and the different interpretations essentially use the probabilities in different ways. So the, the Copenhagen interpretation, or really it's, it's sort of a family of interpretations, is that when we do an experiment, we convert the probabilities that quantum mechanics actually calculates to one of the, one of the possibilities and the more probable ones will happen more often and the less probable ones will happen less often. And so if we do a number of experiments, then they will, then they will obey those laws in that sense. And the problem is that the Copenhagen interpretation needs there to be this barrier between the classical world in which we do the experiment and the quantum world which the experiment is being done of or on. Right? So there's this, there's this, these two different realms, and it's not clear where that distinction should be or why it should be there. Now, Copenhagen interpretation works fine. There's never actually any controversy about how to apply it in real life situations that we've, that we've got to, at least until now. So maybe that's fine. But it leaves people with enough worry that the version of the Copenhagen interpretation that most jobbing physicists actually use isn't really an interpretation. It's what I call in the book and what other people have called shut up and calculate. Which is to say, okay, we know that it's weird that the calculation gives you a probability, but in fact, we still know what it means when we apply it in the sense of, well, you know, things are more likely than not to come this particular way. And if we do lots of experiments, they come out this way. And I'm doing the experiments and I'm classical and the electron is in the box and it's quantum mechanics and the fact that there's some barrier in between doesn't really need to consume us. So that's also unsatisfying for obvious reasons, even though it works. So some more modern interpretations that try to grab the probabilistic bull by its horns, so to speak, are these other ones you mentioned, the many worlds hypothesis, probably the most famous current interpretation of quantum mechanics, and that says that whenever a quantum mechanical interaction happens, essentially that the world splits into a world or a universe or technically the term is probably a branch in which it ha. One, one way that it happened occurs and another branch in which another way that it happened occurs and the probabilities happen because the probability there. There's more probability associated with, you know, the branch that has 80% probability than the branch that has 20% probability. And it is a subjective probability in the Sense that you don't know what branch you're in. Right. So when you're asked, but you know quantum mechanics. So if you've done the experiment before, you do the experiment, you can say, I'm 80% sure that I'm in the 80% branch and I'm 20% sure that I'm in the 20% branch. That's exactly what we mean by subjective probability. But it's conditional probability because it's exactly conditional on the branch structure that you have some way of calculating of the universe. So it does fit in very, very well to my view of probability. And the quote that you had of me, that it's theoretically parsimonious and ontologically extravagant. Ontologically extravagant. So to parse that out a little bit. So theoretically it's parsimonious because the, the object, the mathematical object that I haven't mentioned yet that we calculate in quantum mechanics is something called the wave function. And in, in the Copenhagen interpretation, whenever you make an observation, the wave function is called collapse. It collapses down into just the bit that we saw and then it sort of evolves in a more standard way after that. In the, in the many worlds hypothesis, there's no moment of collapse. There is this branching structure, but that actually is something that is sensible and easily described by the equations of quantum mechanics. So that works just fine. And the probabilities arise because we just don't know which branch we're in. But so that's. So it's parsimonious in the sense that all there is is the wave function. That's great. But it's extravagant because the wave function is this enormous entity that encodes everything that ever could have happened and everything eventually that ever will happen. So, you know, it encodes all of the universes. And there's a, and there is a sense. No, there's not a sense. They really all exist. Right. Yeah. So it's not. So the only world isn't the one where, you know, we're having this conversation. But if, you know, if, if quantum mechanics says that, you know, 14 billion years ago there is a fluctuation in which our corner of the universe never even developed a galaxy, then we're not here in that, in that bit of the wave function. And that really exists and it's really out there. We can't communicate with it in any sense at all, but it's really out there. And, you know, so it's ontological, ontologically extravagant because there's just so much stuff in the world. But we only have access to one little tiny sliver of it. Still could be true. Ontological extravagance is not something that makes the theory fall. It just makes us uncomfortable with it.
Greg McNiff
Absolutely. I mean, your whole discussion of quantum mechanics is fascinating and almost seems as much philosophical as mathematical. But, Andrew, I want to move on to part three, cosmology in a random universe. And as I know, at the beginning, you are a cosmologist and you spend a nice fair amount of time laying out our discovery of the galaxy, our galaxies, other galaxies. And then you introduce an individual, Andrei Sakharov, who has these conditions of the universe. Could you tell us about his conditions and what they may tell us about the beginning of the universe?
Andrew Jaffe
Sure. So Sakharov in particular, he was, he's a very interesting figure. He was at the end of his life, a Soviet dissident. In the middle of his life, he was the Soviet father of the, the father of the Soviet hydrogen bomb. But in his spare time, he was a cosmologist. And one of the questions that bothered him and still bothers us today is why is the universe mostly made of matter and not antimatter? Because our, our theories of the universe, at least naively, are symmetric between matter and antimatter. And so there's no reason to believe, a priori, that one of them should have dominated over the other. But yet we observe the universe today and we don't see. So as many people know, when a bit of matter and a bit of antimatter touch one another, they annihilate, they, they produce high energy photons. But we don't see evidence of this happening very often in the universe at all. And so we're pretty sure that the universe is almost entirely made of matter and not antimatter. And what we don't understand is why. And so Sakharov wrote down three conditions that are required for the universe to make more matter than antimatter. And basically those conditions are, first of all, that you have, you do have to have some asymmetry in the laws that sometimes there must be some reason why a, why a given interaction will produce antimatter, or, sorry, will produce matter more than antimatter. So that's the first condition, the second condition, because actually, at most of the time in the universe, anything that can go forward can also go backwards. So you also have to have a reason why anything that can produce more matter than antimatter doesn't go back and produce and just decay back into what it started with, because then you'd still have the same Amount of matter and antimatter. And then finally you have to have a re. You have to have a way to. This goes back to the thermodynamics we were talking about before. To get out of equilibrium, get out of us. Because in equilibrium, because matter and antimatter by definition in our models has the same mass that in the simplest version, it's very hard. If the universe is hot and cooling slowly, then then no matter what those interactions are, you'd always expect to have the same amount of matter and antimatter. And we think that, that the Big Bang actually does the last thing. Because the universe is not just hot and sitting there, it's hot and expanding and cooling. And as the un, as the universe becomes cool enough, interactions that could happen stop happening. And so if you're able to produce even, even briefly, more matter than antimatter, then you will not destroy it by that, by that expansion. So that's good. So then the question is, do those other things obtain? And the answer is, well, we're not totally sure. Well, they must at some level. If Sakharov is right, and we think he's right, they must happen. But the standard model of particle physics has the seeds of both of those things happening. But actually those, they're, they're really, these are really going back to our model building from earlier on. These are numerical tests we can do on the data to see is the amount of this, of the asymmetry that you would expect to get from the standard model of particle physics enough to produce the asymmetry we see today? And the answer is we're not sure.
Greg McNiff
Yeah.
Andrew Jaffe
And so, you know, and it's kind of more interesting if the answer is no, because that gives us information that there must be something beyond our standalone model, standard model of particle physics. And it's kind of an amazing fact about the cosmological laboratory that we can hopefully make statements about fundamental particle physics just by looking out into the sky and seeing that we are mostly made of matter, not antimatter. I think that's kind of an amazing fact about cosmology today and for the last 50 years that it's become this laboratory not just for doing astrophysics and not just for understanding the world on the large scales, but also a tool for understanding the small scale structure of the world, the particle physics that runs everything.
Greg McNiff
Yeah, you note in the book that these conditions allow us to better understand the universe. And I think to your point, it's only by falling out of equilibrium does the universe build up relics from its past. And you know, out of which comes everything that differentiates it from a featureless gas. So I guess these conditions allow us to, like you said, understand what's going on with the universe and why this disequilibrium. Yeah. Okay. Andrew, after your time, you did your PhD at U. Of Chicago, you moved to the Canadian Institute for Theoretical Astrophysics where you worked with Nick Kaiser and Dick Bond. And you described them as, I think, two of the smartest people. No offense to your graduate advisor, but two of the smartest people you worked with. Obviously, coming from you and given everyone you worked with, that's pretty high praise. And I believe you worked on mapping the cmb, the cosmic microwave background radiation. Could you just talk a little bit about what you did? And then I'm going to ask you to sort of describe the evolution from COBE to WMAP to Planck in terms of the sensitivity of the models or the precision of the models.
Andrew Jaffe
Sure. Let me first say. So, yeah, I work with both Nick Kaiser and Dick Bond, the very unfortunately late Nick Kaiser, he just passed away a few years ago. And with him I worked on more of a variety of subjects. I worked on the way galaxies on large scales are flowing around the universe with him. And I worked on gravitational lensing with Nick, among other things. And it's with Dick Bond who I started working on the cosmic microwave background with. And Dick is, I think he was recently awarded the Shaw Prize in Cosmology. He's multiple prize winner. Like I said, he remains one of the smartest people I've ever met. And he taught me to think about analyzing data from the cosmic microwave background. So he was one of the theorists of the CMB and sort of made predictions for what it should look like. But he was also interested in trying to understand how we would tease out from our data the information that we want. And while I was in graduate school, so before I went to Canada, when I was in Chicago, I wrote a paper lamenting the fact that the fluctuations in the cosmic microwave background had yet to be observed. And so the cosmic microwave background is light from about 400,000 years after the Big Bang, at which time, by understanding the laws of quantum mechanics as they apply to atoms, as we were talking about a few minutes ago, we, we can show from really simple physics, really no physics, no more complicated than those early quantum mechanics people were doing in the, in the 1910s, we can show that the universe went from being opaque to being transparent over time. And as all of the excess light, all the photons that were, that had been bound up in essentially this cloud early on became free. And of course, as we look back, as we look further away, we look back in time. And so if we look eventually far enough away, we see the surface of this cloud, right? And the surface of this cloud is what we call the cosmic microwave background radiation. Now, it's those light, the light is, is here now, right? It's, it's, we're not seeing it now on the cloud, we're seeing it here today. And it's, it's, it's surrounding us. And it takes, it's a baby picture of the universe, right? It comes from a very small fraction of the universe's age when the universe was hot and dense and simple. And there were no objects, there were no galaxies or clusters of galaxies or planets or television sets or anything like that. There was just a gas, an almost featureless gas, but not quite. And it's that almost that's really important. So we knew because the universe was, is lumpy today, that it had to be lumpy back then. And by seeing the slight perturbations from a uniform temperature that were, that we believed then to be present, we could map out the structure of the universe at early times and compare it essentially to the structure of the universe today and see how it evolved from those early days to now. And the COBE satellite, launched, I think around 1989, and with, with results coming out in the very early 1990s, was the first experiment to successfully map out those fluctuations on kind of, with a, on a scale of about 10 degrees on the sky. But that was enough, that was enough to show that this was as predicted by our Big Bang model by, and in particular by particular versions of our Big Bang model, which were consistent with there being a very early rapid expansion of the universe that laid down these small fluctuations that eventually grew to the structures we see today. And this epoch of inflation has become a crucial part of our Big Bang model, even though we don't really understand the details of that. And that's something we're still trying to measure, but with COBE measuring things on sort of 10 degree scales in the early 90s, by the late 90s, I, with Dick Bond and others, were involved in a set of experiments from the ground and more importantly from balloons. So from, from balloons. So not, you know, not just like hot air balloons, but balloons. Well, they are hot air balloons, but not just the kind that you see in the sky taking passengers, but ones that go higher than airplanes, right? And that gets you above a lot of yucky atmosphere that glows and gets, you know, gets you not quite into space, but far enough from, from the dirt of the ground and the glowing sky that you can actually do useful, useful observations of this particular kind. So these experiments called Boomerang Maxima, were the first to start seeing these fluctuations on scales not of 10 degrees, but of a degree and even a bit smaller than a degree. And that's the crucial scale where we started seeing evidence of these things that would eventually grow into the structures we see today, really wholesale. And so we started seeing, we measured this one function called the power spectrum, and we started seeing bumps and wiggles in this power spectrum, as predicted by people like Dick Bond and his collaborator Georgia Stathiw and Jim Peebles and other people who've won all the important prizes in our field because they've come up with these amazing predictions. And by the late 90s, we were starting to see these fluctuations in these balloons. By the early 2000s, the WMAP satellite from NASA at slightly higher angular resolution, but over the whole sky. So with a balloon, you can only see part of the sky. With a satellite, you can see the whole sky. And they saw it with much better than a degree to about an arc minute or so, a bit more than that. And then in the late 2200s, about 2009, the Planck satellite was launched. It was a collaboration mostly between ESA mostly, but also with NASA's input. And it really looked at the whole sky and was able to really successfully disentangle not just the CMB cloud that's at 400,000 light years, but also was able to distinguish it from a lot of the other stuff along the line of sight that also glows that we have to get rid of the galaxies and clusters of galaxies and indeed a lot of gas and dust in our own Milky Way, and at much higher angular resolution and sensitivity. So it really built a beautiful picture of, of the sky as a whole. You can see reproduced in the book in grayscale and online easily in nice beautiful colors, which really started to solidify this model of, you know, it really, you know, this, this, the picture has lots of, lots of fluctuations up and down. And it would have been very easy for our model not to fit these data, but the fit is remarkably good. And we're trying to still tease out lots of information from these data. Planck has essentially given us a standard model of cosmology where we've managed to measure things not to tens of percent, which things were sort of when I was a graduate student or even in about the year 2000 to 1% or a few percent. And that is really a different level of precision and accuracy. Well, we hope it's accuracy. We know that it's precise, we know we have our small error bars. And what we're beginning to test is whether the model in all of its glory and all of its simplicity does fit the data.
Greg McNiff
And Andrew, do you expect the Simmons Observatory to help us answer that question or improve the accuracy?
Andrew Jaffe
So Simons. So it's the Simon. That's all right. The Simons Observatory is in. So it's a ground based telescope. But so to give you, to give you an example, some information about why it's much better than Planck. So Planck was beautiful. It took a long time to build and it had about 50 detectors. So a detector is like one pixel of the CC of CCD on your camera. Right. So that's so. And you know, your phone probably has millions of them and they're much more sensitive than your, than your, than your phone camera. And they're doing different things, but it's the same idea, but it had 50. And the experiments that we're building now, like the Simons Observatory, and it's probably the most powerful, but there's others alongside have hundreds of thousands of such detectors. So it's, the leap forward is absolutely immense. And you pay the price of not being in orbit. So having atmosphere and ground and things to deal with, but you also get the advantage of being able to go fix things when they break. It's very difficult to fix a satellite when it breaks. Hubble Space Telescope was a rare counter example of a satellite we could fix, but essentially that will never happen again. So being able to do something on the ground where you can go and tinker with it and get it to work exactly perfectly is a crucial feature of an experiment. Makes things cheaper, essentially. And so the Simons Observatory will measure not the whole sky. So that's the disadvantage. When you're from the ground, you can only see part of the sky, but it enables you to probe with. Well, we actually have more than one telescope, one which probes with very coarse resolution but very, very sensitively. And it's trying to measure not just the temperature on the sky, not just how bright the light is, but also its polarization. So most people are vaguely aware that light isn't just a brightness, but it has kind of a, kind of a direction. And you're aware of that. If you ever wear polarized sunglasses. Polarized sunglasses. If you tilt your head as you're wearing the sunglasses, you can see that the way things appear changes and you can see really well if you look at reflections off water or something like that. And if you put your head in the right angle you can actually see much more deeply into the water than otherwise. And all light has some amount of polarization and the cosmic microwave background does as well. And some aspects of that polarization trace the early universe in a different way than the temperature, than the brightness of the light. And in particular and I'll just give a technical term, it's called the B mode. It doesn't really matter of the polarization traces, gravitational waves, fluctuations in space and time that came from this inflation that I talked about a few minutes ago. Because we believe that inflation doesn't just produce the bright spots that become the galaxies that we see today, but also produces propagating wiggles in space time that are gravitational waves. Like famously I guess the LIGO Observatory which won the Nobel Prize a few years ago has been observing. Now those are created very locally, but we believe that there should be gravitational waves from the very early universe created by an epoch of inflation. And we hope that that that information about that is frozen into our, our cosmic microwave background polarization. And the Simons Observatory has the power, we believe to actually see it. If it's there now, we might not see it. It might be there but to that below our threshold. But we're hoping that it does see it.
Greg McNiff
Yeah, no, we look forward. That is fascinating. Andrew. That concludes our interview. Thank you so much for your time. You've been very generous. Again, the book really is, does a wonderful job of describing these foundational questions. As I said in my opening, you bring clarity to some of the most difficult concepts in science which still seem like they're evolving or we're learning more almost on a daily or weekly basis. It's been a great privilege to talk to you, really enjoyed it. Thank you.
Andrew Jaffe
Thank you Greg. It's been a lot of fun. Hey, Ryan Reynolds here wishing you a very happy half off holiday because right.
Ad Voice 3
Now Mint Mobile is offering you the.
Andrew Jaffe
Gift of 50% off unlimited. To be clear, that's half price, not half the service. Mint is still premium unlimited wireless for a great price. So that means half day.
Greg McNiff
Yeah.
Andrew Jaffe
Give it a try@mintmobile.com switch up front.
Ad Voice 1
Payment of 45 for 3 month plan equivalent to 15 per month required new customer offer for first 3 months only. Speed slower. 35 gigabytes of networks busy taxes and fees extra. C mint mobile.com.
Podcast: New Books Network
Host: Greg McNiff
Guest: Andrew H. Jaffe, cosmologist and author
Book: The Random Universe: How Models and Probability Help Us Make Sense of the Cosmos (Yale UP, 2025)
Date: November 25, 2025
In this episode, Greg McNiff interviews Andrew Jaffe about his new book, The Random Universe, which explores how probability, models, and inference underpin our understanding of the cosmos. The discussion weaves together physics, philosophy, mathematics, and scientific method, providing a thoughtful yet accessible peek behind the curtain of modern cosmology and how scientists build and judge their models amid uncertainty and randomness.
“Observation is not sufficient. We do not believe our eyes unless we are first convinced that what they appear to tell us is credible.” (05:45, Jaffe)
“Theoretically parsimonious but ontologically extravagant.” (64:51, Jaffe)
Parsimonious: The wave function evolves deterministically, without “collapse.”
Ontologically extravagant: All possible outcomes are realized in some branch; the universe “splits” with every quantum event.
For Jaffe, all interpretations struggle with how to understand probability—he favors viewing probability as conditional on our knowledge and models, including which “branch” we might find ourselves in.
On the necessity of models:
“You can't interpret the world without the model. Right. It doesn't work the other way around. And that's the same thing in science.” (06:33, Jaffe)
On induction and naturalism:
“If you believed that the world was unintelligible...then, you know, literally all bets are off. You therefore believe that anything could happen.” (14:17, Jaffe)
On mathematics and the universe:
“I think the universe is...just one damn thing after another. And...we are the ones who choose to describe that by differential equations and integrals...” (20:29, Jaffe)
On Bayesian inference:
“It’s a model for learning. You start out with some prior information...It transforms your prior into your posterior...that in that sense, it's a model for learning.” (39:53, Jaffe)
On the many worlds interpretation:
“Theoretically parsimonious and ontologically extravagant.” (64:51, Jaffe)
On conservation of humility in science:
“People have done this forever...maybe things like string theory get a lot of press because they seem like, oh, it's a lot of effort on this one thing and maybe it's not right because it seems so beautiful. But this isn't new and it's not. And they're not being foolish. They might be wrong.” (22:46, Jaffe)
For listeners seeking an accessible yet rigorous exploration of the scientific method—as it's genuinely practiced—and the probabilistic foundations of cosmology, this conversation with Andrew Jaffe is an invaluable guide.