
Hosted by LessWrong · EN

[Crossposted from my blog, BlueprintingHeaven.] Please note that the views I held and thoughts I had during the time immediately after my deconversion are not necessarily ones I have right now. It would be boring and annoying to constantly say this in the post, so here is your advance disclaimer! When I tell people I switched from wanting to do astrophysics to wanting to prevent death upon becoming an atheist, they’re usually not too surprised or confused. Sometimes they are, and I might make a blog post on that pipeline later, but most of the time they understand, even if they disagree about ending death being a good thing and even if they knew just how obsessed I was with astrophysics or becoming an astrophysicist. Something that may be more surprising or confusing is my almost total evaporation of fandom behaviour. Back when I was religious, at any given time I was usually into a fandom (more specifically, it was usually a particular character within whichever fandom), and this would be the default topic my mind would drift to in the car, whilst showering, etc. Each fandom phase varied in intensity and length, ranging from obsessively writing pages of [...] The original text contained 2 footnotes which were omitted from this narration. --- First published: May 17th, 2026 Source: https://www.lesswrong.com/posts/4xtmJGeSbCoDEbR2x/how-to-quit-fandom-apostasy --- Narrated by TYPE III AUDIO.

In 1932-33, Soviet collectivization destroyed local farming knowledge and produced a famine that killed somewhere between five and nine million people. It was one of the twentieth century's great tragedies, and James Scott's Seeing Like a State draws a straight line from the ideology that caused it — High Modernism, the belief that society can be rationally reorganized from above — to the disaster that followed. But here's a number that doesn’t appear in Scott's book. Eight billion. That's roughly how many people are alive today, most of them fed by the products of scientific agriculture. Synthetic fertilizers, high-yield crop varieties, mechanized farming. The Green Revolution, which saved millions from starvation in the second half of the twentieth century, was born from the same impulse as High Modernism: it is top-down, science-driven and generic, scaling standardized solutions across entire continents. James Scott's Seeing Like a State is a brilliant book about the former kind of outcome. But it has little to say about the latter.1 This has allowed a generation of readers to walk away with a clean takeaway: Local knowledge good, central planning bad. But that is, at best, half of the story. The question that even Scott [...] The original text contained 2 footnotes which were omitted from this narration. --- First published: May 17th, 2026 Source: https://www.lesswrong.com/posts/iiDzt5qhesmQZiNAj/james-c-scott-seeing-like-a-state --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Many people make costly mistakes when reasoning about their health. Even most doctors make this mistake, because it's not a mistake that's caused by a lack of medical knowledge. Rather, it's caused by a lack of clear thinking. People experience symptoms, and then they look for the root cause of their symptoms. For example, somone with heartburn or pain in their stomach might decide the root cause of their issues is excess stomach acid/GERD (GastroEsophageal Reflux Disease -- a disease affecting around 20% of the population!!). They then treat this root cause with antacids, often for many years. These people sometimes die of esophageal cancer, because the identified root cause was not the true root cause, and the treatment was imperfect. The mistake was identifying a root cause inside the body. The root cause of a health issue is always something outside the body. (In the case of genetic disorders, the root cause is your parents giving you bad genes, and your parents are outside your body.) Our universe is material, and operates on cause and effect. You can always, in theory, trace the causes back far enough to find one that's outside your body. This is [...] --- First published: May 17th, 2026 Source: https://www.lesswrong.com/posts/WiRrHDd3EiaRd6gsu/how-to-reason-about-your-health-issues --- Narrated by TYPE III AUDIO.

Thanks to Megan Kinniment for helpful comments and discussion. TL;DR: Benchmarks like HCAST undersample fuzzy (hard to evaluate) tasks, meaning they might overestimate capability on long-horizon work. To sample fuzzy tasks we need to increase judge capacity: we can either try to build automated judges that match human judgment, or reduce the human effort per grade. To do this, we propose generating fuzzy tasks as a byproduct of real SWE work — snapshot the repo and a proto-spec before starting, and after finishing, use an AI transform to produce an executable spec and LLM-judge conditions. Because the engineer just did the work, verifying the judges or grading the agent directly is much cheaper than grading the task from scratch. I think this would be a good way to collect tasks, as well as a useful personal epistemic tool. This is a two-part series on capability evaluation. Part 1 is about acquiring fuzzy tasks, and part 2 is about analyzing them. Motivation: sampling bias in HCAST There are several well-described limitations of time horizons. But the strongest reason that I don’t update that much on trends in time horizons (and time horizon-like tasks) is because I think all existing evaluations [...] ---Outline:(01:14) Motivation: sampling bias in HCAST(02:47) Making fuzzy tasks sampling viable by increasing judge capacity(04:02) Proposal: sampling from real work(05:18) Advantages(06:10) Discussion(06:13) How inconvenient is this?(06:32) Can we test fuzzy skills by just testing longer tasks? The original text contained 3 footnotes which were omitted from this narration. --- First published: May 16th, 2026 Source: https://www.lesswrong.com/posts/NbDjD47u6WmthgiDC/benchmarking-real-work --- Narrated by TYPE III AUDIO.

(Initially written for the LW Wiki, but then I realized it was looking more like a post instead.) In 1895, the physicist Ignaz Robert Schütz, who worked as an assistant to the more eminent physicist Ludwig Boltzmann, wondered if our observed universe had simply assembled by a random fluctuation of order from a universe otherwise in thermal equilibrium. The idea was published by Boltzmann in 1896, properly credited to Schütz, and has been associated with Boltzmann ever since. The obvious objection to this scenario is credited to Arthur Eddington in 1931: If all order is due to random fluctuations, comparatively small moments of order will exponentially-vastly outnumber even slightly larger fluctuations toward order, to say nothing of fluctuations the size of our entire observed universe! If this is where order comes from, we should find ourselves inside much smaller ordered systems. Feynman similarly later observed: Even if we fill a box of gas with white and black atoms bouncing randomly, and after an exponentially vast amount of time the white and black atoms on one side randomly sort themselves into two neat sides separated by color, the other half of the box will still be in expectation randomized. If [...] --- First published: May 16th, 2026 Source: https://www.lesswrong.com/posts/v8MSczS3CuoqMmTFw/a-relatively-brief-explanation-of-boltzmann-brains --- Narrated by TYPE III AUDIO.

Most of what we currently call "feature discovery" in language models is wrapped up in dictionary-learning methods like sparse autoencoders (SAEs) – which work, and which have been scaled to millions of features on frontier-scale models, but which bundle two distinct commitments into a single training objective: a reconstruction loss and a sparsity loss over a fixed size dictionary. Those commitments make sense if your goal is reconstructive decomposition – if you want to take an activation and rebuild it from a sparse code. They make less obvious sense if your aim is to find interpretable structure (directions? features?) in activation space, to retrieve representative examples, identify causal interventions, or measure how representations change across layers and inputs. And it turns out a lot of that doesn't really need the full SAE machinery. An Exemplar Partitioning dictionary built from Gemma-2-2B L12 activations at p2 (K = 5,129). Left: eight sample regions, each shown with its member count, its exemplar's logit-lens [nostalgebraist, 2020] decode, and an excerpt of a member input with the activating tokens highlighted. Right: a PCA-projected 3D rendering of the Voronoi partition; each cell is one region, with a random selection also labelled with logit-lens decode. This [...] ---Outline:(02:08) Glossary(03:45) Exemplar Partitioning(06:09) Inference(06:51) Properties of the EP dictionary(07:29) Concept detection (AxBench)(08:16) How EP and SAEs relate(09:40) Find and steer refusal(11:37) A free OOD signal(12:45) Cross-checkpoint drift (base ↔ IT)(14:58) Domain saturation(15:59) Inside the partition(17:33) Future work(21:24) Thats all for now --- First published: May 16th, 2026 Source: https://www.lesswrong.com/posts/RroeHBSkBXXDsrryq/an-introduction-to-exemplar-partitioning-for-mechanistic-1 --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Credit: ClaudePlaysPokemon Elevator Shanty by Kurukkoo Disclaimer: like some previous posts in this series, this was not primarily written by me, but by a friend. I did substantial editing, however. ClaudePlaysPokemon feat. Opus 4.7 has finally beaten Pokémon Red, fulfilling the challenge set over a year ago when LLMs playing Pokémon went briefly, slightly viral. Victory Screen! Let's get the throat-clearing out of the way: this doesn't make 4.7 a clear breakthrough in intelligence over 4.6 or 4.5. It's smarter, yes, as we'll discuss below, but not by something one could honestly call a big leap. Rather, step changes have finally accumulated to the point of victory. And to give other models their fair shake: after criticism over its elaborate harness,[1] GeminiPlaysPokemon has beaten Pokémon with progressively weaker harnesses, including about two months ago with a harness comparable to the one Claude uses.[2] As such, this is a bit of a valedictory post, closing off the cycle of Claude playing Pokémon Red, relating anecdotes for the fun of it, and discussing improvements in Opus 4.7, as well as speculating a bit on what this has all meant. Retrospective Anecdotes on Claude 4.5 and 4.6 Our last post, on Opus [...] ---Outline:(01:37) Retrospective Anecdotes on Claude 4.5 and 4.6(06:34) Harness Changes for 4.7(07:52) Improvements (4.6 + 4.7)(08:16) Vision(09:31) Less Tunnel Vision(10:12) Another Level of Spatial Awareness(10:25) Breaks Out of Loops EVEN FASTER(10:55) Victory Road(11:50) The Little Things(12:52) Concluding Thoughts(15:29) Notes on Pokémon as a benchmark The original text contained 10 footnotes which were omitted from this narration. --- First published: May 16th, 2026 Source: https://www.lesswrong.com/posts/sehJYg5Yny9fvpbpt/a-year-late-claude-finally-beats-pokemon --- Narrated by TYPE III AUDIO. ---Images from the article:<img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/v1778911980/lexical_client_uploads/nhzxubdjzolrw4mbkloy.png" alt="Retro video game screenshot showing character fa...

At least we probably won’t have another pandemic. And we still have a partial Jones Act waiver. For now. Small victories. Table of Contents Hanta Hanta I Don’t Wanta. Bad News. Predictions Can Be Easy Even About The Future. Good Advice. The Efficient Market Hypothesis Is False. There Are Four Skills. While I Cannot Condone This. Good News, Everyone. For Your Entertainment. Gamers Gonna Game Game Game Game Game. The Spire Sleeps And So Shall I. I Was Promised Flying Self-Driving Cars. Government Working. Jones Act Watch. Technology Advances. I Said Woo Hoo. Variously Effective Altruism. The Lighter Side. Hanta Hanta I Don’t Wanta We have learned so much less than nothing from Covid. We’re actively stupider. It's 2026, and here we are again, lying about the virus because we are worried that people exposed to it or from the wrong place might face stigma otherwise. Envidreamz: A local news story highlights passengers on the hantavirus stricken MV Hondius cruise ship who are more worried about facing stigma and rejection back home than [...] ---Outline:(00:22) Hanta Hanta I Dont Wanta(04:25) Bad News(08:45) Predictions Can Be Easy Even About The Future(08:55) Good Advice(11:52) The Efficient Market Hypothesis Is False(13:58) There Are Four Skills(16:27) While I Cannot Condone This(20:57) Good News, Everyone(21:42) For Your Entertainment(22:14) Gamers Gonna Game Game Game Game Game(24:09) The Spire Sleeps And So Shall I(29:13) I Was Promised Flying Self-Driving Cars(31:53) Government Working(35:40) Jones Act Watch(41:10) Technology Advances(42:12) I Said Woo Hoo(43:36) Variously Effective Altruism(44:07) The Lighter Side --- First published: May 15th, 2026 Source: https://www.lesswrong.com/posts/gZHNLmHkQ7GjnWsYh/monthly-roundup-42-may-2026 --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Suppose we have a dangerous misaligned AI that can fool alignment audits, and distill it into a student model. Two things can happen: Misalignment fails to transfer to the student. If so, we get a fairly capable benign model.Misalignment transfers to the student. The student might also be worse than the teacher at hiding its misalignment (e.g., due to being less capable). If so, we might get indirect evidence about the teacher's misalignment by auditing the distilled model. In this post, we will discuss the second possibility, which we call incrimination via distillation. Specifically, we propose distillation methods that we hope transfer misalignment without transferring the ability to fool audits, and discuss why these techniques might work or fail. In a future post, we discuss the first possibility, and what distillation methods should be used when aiming to create a capable benign model. We’re excited for research that empirically tests and refines this approach; if successful, this technique could become a valuable part of alignment audits. How incrimination via distillation works Powerful misaligned AI models might not be auditable: they might pass alignment audits but still act on their misaligned drives when given the chance. [...] ---Outline:(01:31) How incrimination via distillation works(02:45) How we propose implementing incrimination via distillation(03:48) Auditability-preserving distillation(04:58) Misalignment-targeted distillation(06:20) Why incrimination via distillation might work(08:02) Why incrimination via distillation might not work(10:38) Conclusion The original text contained 3 footnotes which were omitted from this narration. --- First published: May 15th, 2026 Source: https://www.lesswrong.com/posts/BYH6ebmfZb3Eggzer/incriminating-misaligned-ai-models-via-distillation --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Most technical AI safety work that I read seems to miss the mark, failing to make any progress on the hard part of the problem. I think this is a common sentiment, but there's less agreement about what exactly the hard part is? Characterizing this more clearly might save a lot of time and better target the search for solutions. In this post I explain my model of why alignment is technically hard to achieve, setting aside the regulatory, competitive, and geopolitical challenges, the sheer incompetence and unforced errors of the players, and the other factors which decrease our chances of success. I claim there is something like a "hard core": a common stumbling block for all approaches. In other words, there is something that makes alignment hard, rather than a bunch of unrelated things that make each approach to alignment hard independently. Since many different "hopes" for alignment seem quite far apart, this would seem to be a remarkable and unexpected state of affairs. On the other hand, such an expansive graveyard of failed proposals suggests a common culprit. Semantics. When used informally in conversation, "hard core" is a bit like "complete problem." But there may be [...] ---Outline:(01:32) Background(03:02) The Problem(09:19) The Barrier(16:49) Providing better feedback(22:07) Behaviorist v.s. process feedback --- First published: May 15th, 2026 Source: https://www.lesswrong.com/posts/JT3qCYDimskcBdiEr/the-hard-core-of-alignment-is-robustifying-rl --- Narrated by TYPE III AUDIO.