Invest Like the Best with Patrick O'Shaughnessy Episode: Chetan Puttagunta and Modest Proposal - Capital, Compute & AI Scaling (EP.399)
Release Date: December 6, 2024

In this compelling episode of Invest Like the Best, host Patrick O'Shaughnessy engages in an insightful conversation with two esteemed guests: Chetan Puttagunta, a General Partner and investor at Benchmark, and Modest Proposal, an anonymous investor managing a substantial portfolio in public markets. The discussion delves deep into the evolving landscape of Artificial Intelligence (AI), particularly focusing on the scaling paradigms of large language models (LLMs), the shifting investment strategies, and the broader economic implications.

1. The Evolution of AI Scaling: Pre-Training to Test Time Compute

Chetan Puttagunta opens the discussion by highlighting a pivotal shift in AI development. He explains that AI labs have recently encountered plateauing effects in pre-training scaling, a method where increasing computational power during the training phase directly enhances model performance. This traditional scaling approach, based on power laws, posited that multiplying compute power by tenfold would yield significant improvements in intelligence and capability.

Chetan Puttagunta [05:20]: "We're now shifting to a new paradigm called test time compute... scaling on what's now being called reasoning."

However, Chetan points out that the saturation of human-generated text data has limited further pre-training advancements. As a result, AI development is transitioning towards "test time compute," where models engage in reasoning and solution verification during inference, rather than relying solely on pre-trained data.

2. Implications for Public Tech Companies

Modest Proposal provides a macroeconomic perspective on how this shift affects major public tech entities. He underscores that a significant portion of the S&P 500's market capitalization is now intertwined with AI themes, spanning industries from industrials to utilities. The transition to inference-based scaling aligns expenditures more closely with revenue generation, offering a more sustainable financial model compared to the capital-intensive pre-training phase.

Modest Proposal [11:57]: "If you're a two to five person team, you can take something like coding... a billion dollar trading run is essentially you're committing two funds to do one training run that may or may not work."

He critically assesses leading AI players like OpenAI, Anthropic, and Meta, emphasizing the challenges they face in maintaining dominance amid evolving scaling paradigms and the immense capital required for breakthroughs in synthetic data generation.

3. Shift in Investment Focus: From Model to Application Layer

Chetan elaborates on the burgeoning opportunities within the AI application layer, where small, agile teams are leveraging open-source models like Meta’s LLAMA to innovate without the exorbitant costs previously associated with model training.

Chetan Puttagunta [17:29]: "We're seeing these small teams catch up to the frontier with spend that is not one order, but multiple orders of magnitude less than what these large labs were spending to get there."

This democratization of AI model development is enabling startups to focus on creating specialized applications that deliver significant value, thereby attracting substantial investment at a rapid pace. Modest Proposal corroborates this by highlighting how AI applications are drastically reducing software and human capital costs for enterprises, leading to swift adoption and deployment.

4. Infrastructure and Cost Dynamics

The discussion shifts to the infrastructural implications of moving from pre-training to test time compute. Both guests agree that this transition necessitates a rethinking of data center architectures, emphasizing efficiency and lower latency over sheer computational power.

Modest Proposal [75:54]: "It's clear that we're going to rethink how you want to build your infrastructure to service a much more inference focused world than a training focused world."

Chetan adds that advancements in semiconductor technology, such as Cerebras' efficient inference capabilities, are pivotal in making AI applications more cost-effective and scalable.

5. Valuation Trends and Market Sentiment

Patrick probes into the current investment climate, noting the high valuations of AI startups despite potential competition and market saturation.

Chetan Puttagunta [63:00]: "Our cost of inference is essentially zero and our gross margin for this task is 95%."

Both guests express optimism, attributing the favorable valuations to the drastically reduced costs of AI compute and the surge in demand for innovative AI applications. Modest Proposal emphasizes the resurgence of "Animal Spirits" in the public markets, driven by the transformative potential of AI technologies.

6. Future Outlook and Philosophical Implications

The conversation culminates with speculative insights into the advent of Artificial General Intelligence (AGI). Chetan envisions AGI on the horizon by 2025, driven by advancements in reasoning and autonomous task completion.

Chetan Puttagunta [35:57]: "AGI is very close by... we're very, very close to it."

Modest Proposal raises cautionary notes about the unpredictable nature of AGI development, referencing historical instances where AI systems surpassed human expectations in unexpected ways.

Modest Proposal [84:32]: "Anytime that comes into play, I think the stakes are just higher."

7. Under-Discussed Aspects and Closing Thoughts

Both guests agree that the broader infrastructural and economic implications of AI scaling are under-explored in mainstream analyses. They call for more comprehensive sell-side reports and private market evaluations to fully grasp the transformative impacts of this paradigm shift.

Chetan Puttagunta [82:50]: "We haven't seen sell side reports or analysis on this new paradigm shift... very capital efficient."

In conclusion, the episode provides a nuanced exploration of the current and future states of AI development, investment strategies, and the intricate balance between technological innovation and economic sustainability. Chetan and Modest Proposal offer a forward-thinking perspective, advocating for strategic investments in the AI application layer while acknowledging the challenges and uncertainties inherent in the path toward AGI.

Notable Quotes:

• Chetan Puttagunta [05:20]: "We're now shifting to a new paradigm called test time compute... scaling on what's now being called reasoning."

• Modest Proposal [11:57]: "If you're a two to five person team, you can take something like coding... a billion dollar trading run is essentially you're committing two funds to do one training run that may or may not work."

• Chetan Puttagunta [17:29]: "We're seeing these small teams catch up to the frontier with spend that is not one order, but multiple orders of magnitude less than what these large labs were spending to get there."

• Chetan Puttagunta [35:57]: "AGI is very close by... we're very, very close to it."

• Modest Proposal [84:32]: "Anytime that comes into play, I think the stakes are just higher."

This episode is a must-listen for professional investors, CEOs, entrepreneurs, and business strategists keen on understanding the intricate dynamics of AI scaling and its profound impact on the investment landscape.