Podcast Summary: Pablo Torre Finds Out
Episode: Are You Smarter Than an NFL Quarterback?
Date: December 5, 2023
Host: Pablo Torre
Guests: Alex Smith (former NFL QB), Brandon Ally (co-founder, S2 Cognition)
Episode Overview
This episode dives deep into the NFL's decades-long search to measure "quarterback intelligence" and whether any test can truly predict on-field success. Pablo Torre mixes reporting, interviews, and a bit of self-experimentation as he explores the legacy of the Wonderlic test, the rise of the new S2 Cognition evaluation, and the recent tale of two top rookie quarterbacks: Bryce Young and C.J. Stroud. Torre, joined by former #1 overall pick Alex Smith and neuroscientist Brandon Ally, investigates what these tests actually measure—and whether any of it really matters for playing quarterback at the highest level.
Key Discussion Points & Insights
1. The Choking Panthers, the Rise of C.J. Stroud, and the QB Evaluation Puzzle
- Pablo discusses the disastrous rookie season of Carolina Panthers QB Bryce Young (picked #1 overall), whose intelligence test results were stellar, contrasted with Houston’s C.J. Stroud (#2 pick), whose pre-draft "cognition" score was infamously bad—yet who has played remarkably well.
- Quote: "Bryce Young, the number one overall pick... has been failing every exam and choking repeatedly. ... C.J. Stroud, who they took number two overall... has been awesome." [03:43]
- This contrast sparks the central question: can intelligence tests identify great quarterbacks or are teams just guessing?
2. The History & Problems of the Wonderlic Test
- Alex Smith recounts taking the Wonderlic, a logic-and-reasoning exam, for draft evaluation. He aced it—scoring a 40/50 as a prospective #1 pick (Utah, 2005)—but he and Pablo agree the test has little relevance to quarterback play.
- Quote (Alex Smith): "It has nothing to do with football, Pablo. It's actually like this very logical based..." [11:34]
- Sample question: “Which word does not belong: optician, orthodontist, dentist, optometrist?”
- Quote (Alex): "You got labeled smart guy. ...But depending on your background, where you grew up ... [the Wonderlic] had nothing to do with football." [12:17]
- Pablo reports that starting in 2023, the NFL no longer mandates the Wonderlic at the Combine.
3. What Does Intelligence Mean for Quarterbacks?
- Pablo and Alex break down what a QB actually does play-to-play: interpreting dynamic situations, recalling complex playbooks, making split-second decisions, and handling immense pressure.
- Quote (Alex Smith): "The seconds that take place between me, getting the play call... calling the play, digesting it, reminders... then, now we're talking in like fractions of a second, like microseconds here... your analysis, decision-making and processing." [16:15]
- Confidence, calm under pressure, and the ability to reach a “flow state” are vital—much harder to measure.
- Quote (Alex): "Do you have the guts, the confidence, the calmness given the, you know, the stage to do it all? ...The pressure, like performing under pressure." [17:44]
4. The S2 Cognition Test: The NFL's New Toy
- S2 Cognition replaces the Wonderlic, promising to measure cognitive processing, speed, and decision-making more directly, using rapid, abstract visual puzzles.
- S2 claims its results predict 30% of future passer ratings, whereas Wonderlic explained <0.1%.
- Quote (Brandon Ally, S2 cofounder): "A quarterback's S2 score... predicts roughly 30% of that same NFL career passer rating. ... 30% is enormous." [32:30]
- Yet, the 2023 draft outcome threw this into doubt:
- Bryce Young (Carolina): scored 98/99 (nearly perfect)
- C.J. Stroud (Houston): scored 18/99 (abysmal)
- Stroud went on to have an arguably historic rookie season.
- Quote (Alex Smith on Stroud): "It's... the greatest rookie season we've ever seen—at the hardest position in sports." [23:01]
- Quote (Brandon Ally, S2): "A low score doesn't mean you can't play. ...A high score doesn't mean you're going to be an All-Pro quarterback." [31:07]
5. The Flawed Science—And Human Reality—of Predicting QB Success
- Leaked S2 results shaped public and team perception, but both guests stress that on-field performance depends on countless variables: practice quality, coaching, confidence, and context.
- Quote (Alex Smith): "There's so many ways you can be successful in the NFL, and it's not reliant on one thing like arm talent or decision-making or S2 or whatever." [40:31]
- Both Alex and Pablo do poorly on certain portions of the S2 test, with Pablo comically scoring 3rd percentile in instinctive learning (compared to NFL QBs).
- Quote (Pablo): "Third percentile. That felt like I should be in a home somewhere, like, doing that test." [43:29]
- S2 scores are just one tool among many for front offices, not an answer key—mistakes and surprises remain inevitable.
- Quote (Brandon): "Nobody has a crystal ball. ...If you're going to knock us for being wrong even 20% of the time, we're still, all right, we're still helping make informed decisions." [41:29]
6. The Enduring Mystery of Talent Evaluation
- The episode ends with Pablo reflecting on how failing standardized tests can, counterintuitively, lead to growth and new opportunity—suggesting a healthy skepticism for using any single score as destiny.
- Quote (Pablo): "Sometimes the test you fail winds up becoming one of the greatest things that ever happened to you." [48:23]
Notable Quotes & Moments (with Timestamps)
-
On current NFL quarterback failures and draft misses
- Pablo: "Bryce Young has been dog all season ... but instead he's been failing every exam and choking repeatedly." [02:19]
- Alex Smith: "It's... the greatest rookie season we've ever seen—at the hardest position in sports." [23:01]
-
On the irrelevance of the Wonderlic
- Alex: "It has nothing to do with football, Pablo. It's actually like this very logical based..." [11:34]
- Pablo (mocking the test): "All of which is to say that these questions have a lot to do with quarterbacking." [14:16]
-
On pressure and real QB intelligence
- Alex: "But the most important thing ... is not so much do you have the intelligence and processing, but ... do you have the guts, the confidence, the calmness given the ... stage to do it all?" [17:44-18:03]
- Alex: "Can you lock in? Time slows down almost... in the midst of all of that stuff, external distractions, internal distractions, right from the pressure of the situation and the moment, that it just is—you're unflinching." [19:44]
-
On S2 Cognition and its limits
- Brandon Ally (S2): "A quarterback's S2 score...predicts roughly 30% of an NFL career passer rating. 30% is enormous." [32:30]
- Brandon: "A low score doesn't mean you can't play. ...We’re more interested in 'how does CJ, how does Bryce process information? Nobody has a crystal ball.'" [31:07]
- Brandon, on context: "If you think there is an NFL team out there drafting a player based on S2 alone, you don't know sports, you don't know football. That is just not ever going to happen. S2 is one piece of the puzzle." [33:24]
-
On test anxiety and the S2 experience
- Pablo: "There is a bit of, I don't want to make this all about me...but I experienced a bit of standardized test taking PTSD ...as I was becoming self conscious about what my results were saying about me." [26:47]
-
CJ Stroud’s response to score leaks
- CJ Stroud (archival tape): "I'm not a test, I play football. The people who are making the picks know what I can do, so that's all that matters to me." [37:50]
-
On the unpredictability of quarterback success
- Brandon Ally: "We've been trying to predict human behavior since the beginning of time, and it turns out we're really bad at it." [41:18]
- Pablo: "My general rule of thumb ... is that nobody really knows anything." [41:18]
-
Self-deprecating humor in S2 results
- Pablo: "Your instinctive learning ... you were at the third percentile."
"That felt like I should be, like, in a home somewhere..." [43:29] "I'm going to go home and tell everybody that I outscored CJ Stroud..." - Brandon: "That's very dangerous thinking... you're going to need a whole lot more than an S2 score." [47:39]
- Pablo: "Your instinctive learning ... you were at the third percentile."
Important Segments & Timestamps
- 00:00-04:47: Introduction; Pablo sets up QB flop/success paradox (Bryce Young vs. CJ Stroud)
- 07:57-16:15: Alex Smith on being labeled a "smart" QB, his Wonderlic experience, and what intelligence actually means for QBs
- 20:18-23:01: Discussion of Brock Purdy, undrafted QB outperforming expectations, challenge of scouting "intangibles"
- 26:12-32:30: Pablo takes the S2 test; Interview with Brandon Ally, exploration of S2’s science and limitations
- 31:07-40:23: S2 controversies, validity claims, and the reality that no test is infallible
- 43:02-46:41: Pablo receives (middling) S2 results, with humor and self-reflection
- 47:22-48:23: Final reflections on the limits of standardized tests for life and football
Conclusion
The episode leaves listeners with a sobering, often funny look at the limits of quantifying "intelligence" and "potential" in sport—or anywhere else. The best tests, whether Wonderlic or S2, provide insight in context but deliver nothing close to certainty. The human element—the stuff no test can fully capture—remains at the heart of quarterbacking and talent evaluation.
For more deep sports storytelling, subscribe to Pablo Torre Finds Out.