Hosted by Jeremiah · EN
In recent posts on Trump and dictatorship, people have asked me - how do you know you're not suffering from Trump Derangement Syndrome? I take this seriously; we've all lost loved ones to this condition. The best check on my reasoning would be an objective measure of the health of American democracy. There are several "democracy indices" that purport to do this, but they have a mixed reputation. My impression is that most current accusations of bias are relatively weak - I agree with Claude's analysis here - but they rely enough on "expert" opinion that I don't expect them to convince a skeptic. The newest entrant in this space - Metaculus Democracy Threat Index - works differently, and deserves a closer look. Metaculus is a prediction site - like a prediction market, except that no money changes hands. People can record their guesses for how future events will turn out, which get aggregated by an algorithm (currently just a recency-weighted median, although they've done fanicer things in the past). Their Democracy Threat Index is a collection of 153 questions relevant to US democracy. For example: https://www.astralcodexten.com/p/the-metaculus-threat-to-democracy
The most controversial part of last week's article on the Midjourney ultrasound scanner was medical experts' recommendation against whole-body screening (including existing whole-body screening technology using MRI). Isn't this crazy? Whole-body screening can save lives by detecting serious diseases like cancer. The experts counterargue that it finds so many false positives - minor zit-like imperfections that would never have caused problems, but which cost patients time, money, anxiety, and side effect burden to investigate - that it ends up net negative. But isn't this just a problem of setting thresholds correctly? Can't you commit to only investigating the most obviously bad things, then ignore the rest? This seemed like an interesting problem to investigate in more depth, so I've tried to get numbers. These are rough estimates loosely based on parameters extracted from unsatisfactory studies1 - please don't take them seriously as exact values, just as right-order-of-magnitude estimates. We'll focus on whole-body MRIs, since this is a well-studied existing technology, then speculate later on how the results might generalize to whole-body ultrasound. https://www.astralcodexten.com/p/should-people-avoid-whole-body-screening
like that, except from a medium-sized startup instead of a tech giant. Earlier today, they announced a pivot to medical scanners. The new MidJourney Scanner, which they describe as "a bold new kind of machine to reimagine the foundations of healthcare and our relationships to our bodies", will be a tank of water surrounded by a ring of ultrasound scanners. The patient goes into the tank, the scanners emit ultrasound from all angles, and then some fancy AI reconstructs the echoes into a 3D picture of the body. The result is ultrasound tomography: the same sort of rich data as a CT or MRI, but done via ultrasound, with no harmful radiation, in twenty seconds. This is cool, and it's great to be ambitious, but I think the narrative among the SF AI crowd has escaped its basis in the medical facts, so I want to throw a bit of cold water on it. I'm a psychiatrist, which is about as far as you can get from radiology while still being a doctor, so this is speculation only, and you can ignore it if you find an actual radiologist or ultrasonographer with opinions. Still, my take is that this scanner isn't useful for most current serious medical applications. It could potentially be used to pioneer a new class of low-risk screening applications, but it's unclear whether these are good, and depends a lot on what other future technology gets invented in parallel. https://www.astralcodexten.com/p/preliminary-thoughts-on-the-midjourney
In 1917, three children in Fatima, Portugal claimed to have seen the Virgin Mary. They promised she would perform a miracle on a certain day in October. Nearly 100,000 pilgrims arrived, hoping to see whatever happened, and nearly all report that the sun turned pale, changed color, and spun around. Many other writers have investigated the children and their visions, but I was fixated on this sun miracle. Despite popular discussion of "mass hallucinations", this is AFAICT the only example of tens thousands of people all saying they witnessed the same impossible thing, at the same time. I got kind of obsessed with this; you can read my preliminary investigations in this, this, and this post. One of the first things I found was that there were many other sun miracles - at least ten! - similar to Fatima. Most were associated with Marian apparitions, but one was at a Buddhist temple. Bigfoot only gets sighted by lone hikers; ghosts are only ever in the corner of your eye; UFOs are just blurs in the sky. Of all the countries and outposts in the vast empire of the unexplained, it's only this one phenomenon - the spinning, multicolored sun - that regularly gets seen by thousands of people at once, in broad daylight. Speaking of "regularly", there's one spot where it continues even today. Fifty years ago, the Virgin Mary appeared to six children in Medjugorje, Bosnia. Now those children are well past middle-age, but she continues to come. Three of them report that she's appeared less frequently as the years go by, but the others still see her every day at 6:40 sharp. Travelers to Medjugorje, especially those passing through around 6:40, report a slew of miracles, including the spinning sun. Certainly this is true of those whose hearts are pure. But even the atheists get lucky sometimes. I was shocked never to have heard about this before. There's a place you can just go, and have a decent chance of seeing a real miracle? People take vacations to the Bahamas for the beaches, when they could go instead to Medjugorje and see the natural law of the universe get violated in real time? Seems crazy! So in early April, I and my extremely-accommodating, long-suffering wife flew to Dubrovnik, rented a car, and drove down a series of windy mountain roads toward the Bosnian border, hoping for a miracle. https://www.astralcodexten.com/p/waiting-for-the-miracle
Guest post by Alexander "Sasha" Putilin [This is a guest post by 2024 ACX grantee Sasha Putilin. I encourage any ACX grantees who are interested to write about their projects. - SA] The results of my ACX Grants 2024 project are in. The project attempted to replicate the 2023 study "Learning at your brain's rhythm: individualized entrainment boosts learning for perceptual decisions". It claimed that if you read a person's brain waves, figured out an individual peak alpha frequency, and flashed a bright white light at that frequency, then they learned a certain perceptual task faster. Why bother? The result hinted that learning may depend in part on how well the brain keeps its rhythms coordinated. In other words, perceptual learning may rely on an internal brain metronome. If flickering light could act as an external metronome, it might help the brain maintain the right rhythm and learn faster. The study offered an invitation to develop new frontiers of neuroscience and biohacking. If the effect generalised to other types of learning, you could build a learning helmet: put it on your head, let it read your brainwaves, flicker light tailored to your individual brain — and you learn a new skill quicker. And no, it didn't replicate. Most likely it can't replicate, because the effect is probably not real. The original study obscured the data with summary statistics. Running a $32,000 replication was excessive. We could've caught the issue with this study if we simply looked at the original data carefully. *record scratch* *freeze frame* Yep, that's me. You're probably wondering how I got here. Here's the story. https://www.astralcodexten.com/p/never-cross-a-river-four-feet-deep
I recently had a minor spat over someone misinterpreting my AI beliefs (see section marked "Update" at the bottom here), so I thought I would list them in one place, so I can refer people when they ask. Timelines1 Define AGI as AI intelligent enough to do 90% of knowledge work jobs. I think there's a 25% chance of AGI by 20272, a 50% chance by 2034, and a 75% chance by 2045. Basic argument: In a certain sense, AI is already "smart" enough for this (eg it can answer quantum physics problems, which require higher IQ than most knowledge work). Its remaining limitations are that it's confused, unagentic, lacks situational awareness, and tends to hallucinate. The METR time horizon graph, and several other related benchmarks/experiments/intuition pumps, suggest it's improving on time horizons at an (exponential) rate that lets it cross human-level performance sometime around the early end of the schedule above, and subjectively it feels like harder-to-measure constructs like situational awareness are improving about as fast. Arguments for earlier: recursive self-improvement causes a speedup compared to the trend. This is one of the biggest blank spots in my model: I don't know how fast RSI will progress, and I don't think anyone else does either. There's some function mapping a combination of AI talent and compute to progress, and we don't know how it behaves in the domain when there's far more talent than compute available. It could fizzle out completely for lack of compute, or it could go vertical. The AI Futures Project has done some of the best work trying to model this, but even they have low confidence.
The philosophers of the Frankfurt School practiced a technique called negative dialectics, where concepts are defined as much by what you can't say about them as what you can. Appropriately, the Frankfurt School has ended up defined by what you can't say about them. You can't say that they invented a new form of left-wing thought called Cultural Marxism. This would be (according to Wikipedia) the Cultural Marxism conspiracy theory, a "far right anti-Semitic conspiracy theory that misinterprets Western Marxism, especially the Frankfurt School, as being responsible for modern progressive movements, identity politics, and political correctness". You're not supposed to dub them a transitional stage between Communism and postmodernism. You're not allowed to speculate that a lot of the academic humanities, as they're practiced today, descend from the Frankfurt School's brand of critical theory. You're not supposed to think of them as the point where the muscular pro-technology leftism of the early 1900s shattered into the pessimistic degrowth leftism of the present. Art is long, life is short. Most of us only manage to not do a few things in our limited span on Earth. But the Frankfurt School managed to not invent so many movements - to not be involved in so many of the crucial ideological shifts of the past century - that they caught my attention. Who were these people? What other aspects of our culture might we be unable to say they were involved in? For answers, I turned to the classic history of the group, Martin Jay's The Dialectical Imagination. The basics are simple enough: the School was founded in Frankfurt in 1923. It attracted great philosophers like Max Horkheimer, Theodor Adorno, and Herbert Marcuse. When the Nazis took power in the early 1930s, the mostly-Jewish Frankfurters fled to America, where friendly locals helped them continue their work in affiliation with Columbia University. Mid-century Americans were suckers for sophisticated European intellectuals, and when the rise of fascism and World War II started dominating headlines, the German-Jewish Frankfurters were natural experts to help Americans process the situation. By the end of the war, they were firmly established as thought leaders. Some - including Horkheimer and Adorno - returned to Germany to rebuild its intellectual culture from the ruins; others stayed in America and remained relevant through the 60s and 70s. But figuring out what the Frankfurters believed is more complicated. Forget about the thin line between universally-acknowledged fact and fascist conspiracy theory. The School itself was famously coy, worrying that if they explained themselves too clearly, people would caricature their beliefs and integrate them into the existing capitalist system. Even when they did speak "clearly", it was in the sort of German philosophical register where "the negation of the negation" is a totally normal thing to say. Having only read a single book on them, I will no doubt fall into all the failure modes that they and their successors warned us against. But here are the analogies, intuition pumps, and parables that I found helpful. https://www.astralcodexten.com/p/book-review-the-dialectical-imagination
I'm not saying AI is superintelligent or can decide better than you can. I'm saying that if you - like me - spend an hour or so doing research before voting on local seats, AI can aid that research very effectively. And if you don't do that research - because you weren't willing to waste an hour on it before - AI makes it so much faster that you might want to start. I gave Claude a prompt something like (edited for coherence): I'll be voting in the June 2026 California primary. I'm a centrist liberal abundance YIMBY whose favorite political writers are Kelsey Piper, Matt Yglesias, and Ezra Klein. I'm wary of government overreach, but I'm not a doctrinaire libertarian and want to help people when we can figure ways to do it that work. I'm going to ask you about each race on my ballot, and I'd like for you to list the various candidates' bios, policies, endorsements, your read on the most important differences between them, and your advice for me as I try to make my choice. …and got back answers like the following: https://www.astralcodexten.com/p/use-ai-this-election
One popular objection to AI concerns is to declare that LLMs can never be AGI. You need a "new paradigm". Therefore, AGI is so far in the future that it's not worth worrying about. A common counterargument is to claim that no, LLMs can become AGI. But even without that counterargument, I think the "therefore" fails on its own terms. The key question is: how much of a new paradigm do we need? The landmark discoveries on the road to modern LLMs are something like: 1950s: Neural networks 1967: Multi-layer perceptron 2010: Modern deep learning 2017: Transformer, LLM 2022: RLHF, chatbots 2024: Chain of thought / test-time compute We can think of this as an "evolutionary tree", where a given LLM (let's say Claude Opus 4.7) shares a recent "common ancestor" with all other chatbots, and only a very distant "common ancestor" with everything else descended from the multi-layer perceptron. If AGI needs a "new paradigm", what common ancestor can we expect AGI and LLMs to share? AGI will very likely use neural networks, because the human brain is a neural network and qualifies as an AGI. It will probably use deep learning, because although deep learning isn't exactly analogous to the brain, it seems like a pretty reasonable way to emulate the brain's learning algorithms onto computer hardware. Skeptics like Yann LeCun and Gary Marcus usually pinpoint LLMs/transformers as the step where we went wrong. LeCun thinks that the first AGIs may be within the deep learning paradigm (but not LLMs); Marcus thinks that they'll combine insights from deep learning with something else. How soon should we expect a new paradigm as revolutionary as LLMs/transformers? Since we got LLMs/transformers nine years ago, Lindy's Law suggests nine more years. How soon should we expect a new paradigm as revolutionary as deep learning? By the same logic, sixteen years from now. Lindy's Law has a heavy tail, which means we can't simply halve these to find our 25th percentile estimate. Our 25th percentile estimate for the next advance as exciting as LLMs should be three years from now; for deep learning, it's five years. So even if you think AGI will require a further paradigm shift as big as the invention of the LLM or as deep learning itself, you should have 25% chance it will be developed in the next 3 - 5 years. Which is about as long as the LLM-only crowd think things will take! This isn't an excuse for relegating the risk of AGI to some vague indefinite future. It could still be the late 2020s or early 2030s! (Might we expect that low-hanging-fruit effects make the next paradigm harder to find than the last one? In practice, fields get more researchers as time goes on, and that effect usually causes time-between-advances to be approximately constant. And in fact, the number of AI researchers has grown at an unprecedented pace for a scientific field, and growth will enter an even faster regime once AIs themselves can contribute. Overall these make me think things will go even faster than Lindy's Law predicts - but I think Lindy's Law is a useful upper bound.) (Would there still be a long time between the invention of the new paradigm and the point where it could be used to maximum effect? It took five years between the invention of the transformer and ChatGPT, the first commercially-successful transformer-based project. But most of that time was spent scaling up, and we've already scaled up. If we invent a new paradigm in 2030, then any frontier lab willing to bet on it can quickly provide it with levels of compute sufficient to train human-brain-sized models.) This is my attempt to talk to the new-paradigm-wanters in their own language, but I think there's also a subtler point that undermines this worldview. In the past, new paradigms have proven useful in allowing scaling to continue after an old paradigm passed the regime where it could efficiently convert scale to results. LLMs still seem to be able to convert scale to results; while this continues, new paradigms won't be necessary, and frontier labs won't risk pursuing them. If scaling ever hits a wall, there will be a few months of confusion as frontier labs look over various new-paradigm-proposals that they already have lying around, and throw them at the wall to see what breaks through. Then scaling will continue from wherever it left off. The best way to forecast future AI progress is to extrapolate from current LLM scaling. This should work if LLMs scale all the way to AGI. But it may also work even if they don't. First, because we might get the new paradigm so soon that it won't be a significant source of delay. And second, because the most likely place for a new paradigm to start is wherever LLMs stop working, going at the same rate. https://www.astralcodexten.com/p/new-paradigms-wont-save-you
Sorry, I give up. In past elections, I've covered every single candidate for governor of California, from the incumbents all the way down to the cranks. In 2022 there were twenty-six of them, and I covered them all. But sorry, I give up. This year there are sixty. It's too many. I can't disambiguate them all into unique individuals with their own personalities, hopes, and dreams. So as consolation for the list I'm not giving you, here are the basic types, and a few examples of each. The Top-Tier Democrats One of these people will definitely win, but what else is there to say about them? They're all the same. They've all paid the danegeld to some set of unions and interest groups, then put up some kind of incredibly generic platform about how they're compassionate but also a fighter. I can't bring myself to name any of them or discuss them further. The Top-Tier Republicans More or less the same as the top-tier Democrats, minus the chance of winning. They all have cowboy hats and flag pins. They all pose on horseback at their ranch. They all promise to Take Back California for the forces of America and its god, Donald J. Trump. None of these people are actually interesting, but honorable mention to Sheriff Chad Bianco, whose name is Italian for "white Chad". He might be the perfect Republican candidate: https://www.astralcodexten.com/p/the-types-of-candidate-you-find-in