Odd Lots: Jack Morris on Finding the Next Big AI Breakthrough

Podcast: Odd Lots (Bloomberg)
Hosts: Joe Weisenthal & Tracy Alloway
Guest: Jack Morris, AI Researcher, Cornell PhD, formerly with Meta
Date: September 26, 2025

Episode Overview

In this episode, Joe and Tracy are joined by Jack Morris, a PhD candidate and AI researcher from Cornell (with industry experience at Meta), to dig into the current state and future direction of AI research. The conversation moves beyond the business headlines and discusses the science driving foundational AI advances, the challenges of evaluating progress, and the hidden factors that might shape the industry’s next big breakthrough. They also explore the research ecosystem, the importance of datasets and hardware, the industry gap between China and the US, and the philosophical and practical complexities of working in cutting-edge AI.

Key Discussion Points and Insights

1. The Incremental Nature of AI Progress

• Diminishing Returns from New Models:
  • Joe and Tracy express skepticism about the transformative power of the latest large language models (LLMs), like GPT-5, compared to earlier versions.
  • Joe: "I use ChatGPT every day. It does not strike me as like obviously better for my uses than like the O3 models... It did not strike me as... an amazing... step function or whatever." (03:01)
• Jack’s Perspective:
  • He agrees progress feels incremental now. The “wow” factor has faded, but improvements can be unpredictable.
  • “It gets faster for a little while and then it feels like things have slowed down... progress is never quite in the areas that you expect.” (15:13)

2. Public vs. Private AI Research

• Hybrid Pathways for Researchers:
  • Jack explains the blurred lines between academia and industry, especially in AI, where private labs often lure top talent:
    • "All the academic institutions do public research... AI labs like OpenAI, Anthropic... do private research... A cool thing about getting your PhD right now is you can do research, write about it, and then publicize it...” (04:18)
• Transparency and Research Output:
  • Once you join a closed lab, public commentary and publishing typically stop.

3. What Do AI Researchers Actually Do?

• Research Activities:
  • AI research is highly specialized, including areas like speeding up models, refining hardware, curating data (training material), and advancing training algorithms.
  • Jack’s focus is on bringing the rigor of information theory into understanding how many 'bits' or how much information LLMs store. (06:28)

4. How Is AI Progress Evaluated?

• Benchmarking with Data Sets and ELO Scores:
  • Most evaluation is done via standardized datasets (e.g., SWEBench for coding, International Math Olympiad for math):
    • "When a new model comes out, they can say, oh, look, we actually got a higher score on [SWEBench]." (08:13)
  • For creative tasks, models are tested head-to-head and humans rank results—using ELO scores, much as in chess. (09:38)
  • Joe: "The algorithm made famous in the social network... still the workhorse model for comp evaluation." (10:30)
• Benchmarks vs. Real-World Usefulness:
  • Benchmarks can be misleading and don’t always capture “je ne sais quoi” qualities (like model “personality”) or everyday reliability.

5. AI Improvement Methods: Supervised Learning vs. Reinforcement Learning

• Supervised Learning:
  • Models mimic examples from internet or curated data sets (“the best it could do is emulate Reddit posts very well”). (13:15)
• Reinforcement Learning (RL):
  • Now increasingly prominent—lets the model try things, then rewards success, leading to breakthroughs like the O3 model’s advancements in math:
    • “They found out a way to kind of let the model think for a while and then give it a reward... It's kind of scary.” (14:13)
  • Tracy, humorously: “Take a cookie, paying robot...” — Jack: “You just give it a higher number and that makes it happy.” (14:28–14:41)

6. The Year of Agents—and What Didn’t Happen

• Hopes for ‘Agents’:
  • 2025 was expected to be the year LLM-powered agents would become practical for real-world tasks (calendar management, workflow integration).
  • Reality: “The agents are still pretty bad, the ones that you can use. But they did get way better at competitive math...” (15:13)
• Barriers:
  • Gathering environment-specific data and seamlessly integrating language models with application ecosystems is still a huge challenge.

7. Data as the New Differentiator

• The Data Race:
  • Quality and exclusivity of datasets increasingly create strategic moats.
  • “Public data sets... pretty much used to train O3 or GPT5... Apparently there is a much larger amount of private data than public data.” (23:36)
  • Examples: xAI has proprietary Twitter data, Google has YouTube, Anthropic has a pipeline of scanned books, OpenAI has user conversation data.
• China’s Advantage via Open Sourcing and Scraping:
  • Chinese labs can freely scrape and train on all sorts of content without copyright constraints, potentially leapfrogging American labs in some areas.
  • “The Chinese model makers can just sort of take all the books that they can pirate from the Internet and train on them and they're not violating any laws... Which is honestly great for us because then people like me could probably download a model that's better than we would get otherwise.” (27:17)

8. Open Source vs. Closed Source Models

• Jack on Open Source:
  • Open source accelerates the field and helps ensure continuity—for users (e.g., the “AI boyfriend” issue after model updates) and researchers— but also leaks proprietary innovations and secrets.
  • “If you ever fall in love with a model, you should fall in love with an open source model.” (38:10)

9. Personalization and Online Learning: The Next Big Thing?

• Jack’s Prediction:
  • The next breakthrough may come from models that adapt to individuals (personalization) and continuously improve (online learning), rather than occasional “one size fits all” updates.
  • “My personal opinion is the next round of improvements in AI models will come from some type of personalization and online learning...” (33:18)
• Current Limitation:
  • Models are static; personalization is shallow and resets with major version changes.

10. Economic Value vs. Technical Cutting Edge

• Is Progress Chasing Business or Science?:
  • Jack emphasizes that economic impact does not require solving technical “AGI” (Artificial General Intelligence):
    • “AI could be economically transformative without having a higher ceiling... It needs to be more consistent or like dependable, than actually smarter.” (36:10)

11. Researchers’ Motivation: Mission, Money, Ego

• Salary Temptations in AI:
  • Life-changing salaries (tens of millions per year) are real—but not necessarily decisive.
  • “I honestly think personally the marginal difference between having like 10 and 20 million dollars is like very low...” (41:56)
  • Many researchers still prioritize working on important, challenging science and having an impact on the field or society.
  • Titles, mission, and the opportunity for breakthroughs matter as much as compensation.

12. China–US Gap: Data, Compute, Talent

• Is the Gap Stable?:
  • Despite DeepSeek’s buzz, Jack sees a 6–12 month lag for Chinese labs due to limitations in compute, talent, and unique data, but also notes that US labs haven’t seen much migration of talent to China.<br>
  • American labs may have an enduring lead, especially with proprietary data-acquisition arms like Anthropic’s “container-loads of books” operation. (31:43)

Notable Quotes & Memorable Moments

• On Incremental Progress:
  • “It gets faster for a little while and then it feels like things have slowed down and the progress is never quite in the areas that you expect.” — Jack (15:13)
• On Reinforcement Learning:
  • “The way they did that is actually through reinforcement learning... They found out a way to let the model think for a while and then give it a reward...” — Jack (14:13)
• On Benchmark Limitations:
  • "It does not speak to me of a line towards something we would call anything resembling human intelligence." — Joe (12:45)
• On Proprietary Datasets:
  • “If there is a really good website that should have been scraped and downloaded into the model, it should probably be used. But there apparently is a much larger amount of private data than public data.” — Jack (23:36)
• On Open Source and Model Upgrades:
  • “If you ever fall in love with a model, you should fall in love with an open source model.” — Jack (38:10)
• On Salaries vs. Mission:
  • “...there's more of a desire to be there the next time something really interesting happens that supersedes the money.” — Jack (42:30)
• On Model Personalization:
  • “...instead if you had a chatgpt that's specific to Bloomberg or specific to your work, it might be able to use more of its brain to do work for you.” — Jack (33:18)
• Tracy’s Summary:
  • “One thing I thought was very interesting was this idea that everyone gets excited about a specific improvement in AI and then it seems... that particular one doesn't materialize and instead something else emerges as... the big breakthrough.” (46:07)

Timestamps for Key Segments

• Intro Banter on New Models: 01:44–02:51
• Introducing Jack Morris and the Academic/Industry Divide: 03:32–05:10
• What AI Researchers Do & Jack’s Niche: 05:10–07:18
• How Models Are Evaluated: 07:53–09:38
• ELO Scores, Personality, and Evaluation Limits: 09:38–10:55
• Training Data & Model Limitations: 11:25–12:43
• Supervised vs. Reinforcement Learning, The O3 Jump: 13:15–14:41
• Year of the Agent Fizzles, Why Agents Are Hard: 15:13–17:20
• Data as the New Moat / Data-Driven Competition: 23:36–25:30
• China’s Open Model Surge and DeepSeek: 27:06–29:32
• Researcher Incentives (Money, Mission, Ego): 41:04–44:16
• Personalization and Online Learning as Next Breakthrough: 33:18–35:34
• Open Source Pros/Cons, Model Upgrades and User Continuity: 38:10–39:23

Overall Takeaways

• AI progress is both dazzling and incremental—major “wows” are fewer, but underlying methods like reinforcement learning signal more fundamental shifts.
• Data, not just algorithms, now defines the leaders. Proprietary, platform-specific, or hard-to-acquire datasets make the big labs hard to catch.
• Open source AI is a double-edged sword: It democratizes improvement but also risks proprietary advances and “leaking” sensitive data.
• Personalization and continuous online learning are likely the next frontiers, enabling models to truly rival or surpass real human professionals in terms of “getting better” over time.
• Massive money is now routine for top researchers, but mission, culture, and the chance to make history still matter.
• International competition (notably, China/US) is fierce, but the US retains key leads—for now—in data, compute, and talent.

For Further Engagement

• Follow Jack Morris: @jxmnop (Twitter)
• Odd Lots newsletter and episodes: bloomberg.com/oddlots
• Community chat: Discord.gg/Oddlots

This episode offers a grounded, up-to-the-minute exploration of what really drives advances in AI—the science, the incentives, the ecosystems, and the messy, unpredictable nature of discovery and innovation.