Thanks to @Jeremy Gillen for reading and commenting on the draft. This was written while I was was funded by the Advanced Research + Invention Agency (ARIA) through project code MSAI-SE01-P005.

I have tried to achieve two goals in this post. The first is to provide a self-contained explanation of Natural Latents using lots of pictures of probability distributions. The second is to frame Natural Latents in terms of statements about mutual information, rather than the KL-divergences and Bayes nets that Wentworth and Lorell normally use[1]. The two approaches are mathematically equivalent, but the different framings can bring slightly different way of thinking about the problem.

This post will focus on what it means for a variable to satisfy the natural latent conditions and what those conditions correspond to intuitively. I'll also discuss a little bit about the motivation for studying natural latents. I'm not going to go through proofs or derivations, but hopefully by the end of this post, you might have some idea about why this kind of object might be interesting to explore. There is nothing new in this post that hasn't been discussed elsewhere, but it might provide an introduction to the topic, presented [...]

---

Outline:

(01:34) Motivation: Natural Abstractions

(10:37) Introduction

(14:54) The (Exact) Natural Latent Conditions

(14:59) The Exact Mediation Condition

(17:33) The Exact Redundancy Conditions

(19:22) More exact Natural Latent examples

(22:46) Approximate Natural Latent Conditions

(23:16) Approximate Mediation

(25:26) Approximate Redundancy

(28:00) Introducing Randomness to Latents