Podcast Summary: No Priors – "From Job Displacement to AI Trainers, Brendan Foody on Work in the AI Age"

Date: April 10, 2025
Guests: Brendan Foody (Co-Founder & CEO of Merkor)
Hosts: Elad Gil & Sarah Guo

Overview

This episode explores how artificial intelligence is revolutionizing the world of work and hiring, featuring Brendan Foody, CEO of Merkor—a company at the center of the shift, recruiting individuals who help train advanced AI models for top labs globally. The hosts and Brendan discuss the transformation of the labor market in the age of AI, shifts in data collection, how models outperform humans in talent evaluation, the looming issue of job displacement, and the future roles of both people and AI.

Key Discussion Points & Insights

1. What Merkor Does and the AI-Hiring Revolution

• Role of Merkor

  • Merkor uses large language models (LLMs) to automate talent assessment, replacing human resume review and interviews.
  • Their models are now widely used by leading AI labs globally to hire people that then help train next-gen AI (00:47).
  • The company evolved from a general hiring platform to focusing on vetting highly capable individuals for model training and evals (01:57).

• Foody on the Transition in Human Data Markets:

  • "It was transitioning towards this vetting problem of how do you find some of the most capable people in the world that can work directly with researchers to push the frontier of model capabilities." (01:57, Foody)

2. AI Outperforming Humans in Talent Evaluation

• Superiority of AI in Hiring

  • Merkor observes their evaluation models are already exceeding human hiring managers at predicting talent in most domains (02:57).
  • Foody predicts it will soon be “irrational to not listen to the model” (02:57).

• Identifying Outliers & the Power Law of Performance

  • Models can detect "10x" performers, revealing a power law in knowledge work outcomes (03:44).
  • "The power law nature of knowledge work frames the importance of performance prediction." (03:44, Foody)
  • Distribution of ability and effectiveness varies by field—investment is very power law, manufacturing less so (04:29).

3. Domains Where AI Outperforms (or Not)

• Text-Based Domains as AI’s Sweet Spot

  • If a task can be measured with text, models are suprisingly effective (05:21).
  • Multimodal or “human” signals—like passion or sales ability—remain more difficult for models (05:21).

• Volume Matters

  • High-volume, standardized hiring (e.g., for similar roles) is easier to automate; complex, varied roles are harder (06:14).

4. Discovering Hidden Signals and Improving Matching

• Unlocking Online Signals

  • “One of the really interesting things for engineering is that there's so much signal about a lot of the best engineers online that I don't think people properly tap into.” (07:16, Foody)
  • Looking at GitHub, personal websites, and more offers untapped info.

• Non-Obvious Predictors in Other Fields

  • International students studying in Western countries tend to collaborate/communicate better—a pattern discovered through data, not intuition (08:26).

5. Evaluating Models vs. Human Performance

• Prediction of Displacement

  • Foody warns of rapid, painful job displacement in many fields (09:58).
  • He anticipates a large political and populist backlash, and a need to rethink reallocation of wealth and work (09:58).

• Jobs More Resistant to Displacement

  • Physical world roles (e.g., robotics data collection, service jobs, therapy) will linger longer compared to digital/knowledge work (11:04).

6. Skills & Adaptability in the AI Age

• Resilience & Fast Learning

  • Quoting Sam Altman, Foody emphasizes versatility and rapid learning over specializing in areas soon to be automated:
    "You just need to be able to navigate that quickly." (11:53, Foody)

• Which Skills Are Models Fastest To Learn?

  • Tasks with verifiable feedback loops (math, code) are automated most rapidly (12:22).

• Generalization Limits

  • While models generalize well in math/code, new domains still require domain-specific data and transfer learning (13:41).

7. The Future of Evals and Assessing Model Capabilities

• Evaluation Crisis

  • As AI models reach and surpass human benchmarks, it’s harder to tell them apart or assess superhuman ability (13:59).
  • The need shifts from zero-shot/academic exams to evaluating “economically valuable work” (14:37).

• Industry-Specific, Task-Oriented Evals

  • Foody encourages starting eval-building in homogeneous, repetitive fields (e.g., customer support) and anticipates years-long efforts for complex, creative roles (15:34).

8. Shifting Role of Humans: AI Trainers and Beyond

• Rise (and Fall) of Data Collection Jobs

  • “It would not surprise me if that becomes the most common knowledge work job in the world.” (19:17, Foody)
  • Question: is training your replacement a six-month job, two-year job, or longer? Foody notes there will always be a new "frontier" until full automation (20:00).

• Human Evals As Essential Inputs—For Now

  • AI may eventually bootstrap its own eval criteria, but domain expert input remains crucial for grounding (21:44).

9. Economic, Legal, and Social Implications

• Universal Basic Income of a Sort?

  • Many sectors already act as de facto UBI (public sector, academia, big tech), suggesting the “surplus” from AI gains could be distributed (27:10).

• Will AI-Evaluated Layoffs Become Illegal or Inevitable?

  • Foody predicts economic incentives are too strong to prevent widespread adoption, despite possible pushback (28:31).

• AI as a Manager, Not Just Contributor

  • Future AI could outperform human managers in breaking down problems, assigning tasks, and managing performance (29:14).

10. Building for the Next Phase at Merkor

• Key Near-Term Goals

  • Grow the network of top global talent (the "supply side") by offering free tools and a compelling experience (33:29).
  • Reinforce a "data flywheel" by learning from employer feedback on candidate performance (34:46).

• Marketplace of Human and AI Agents

  • Envisions a labor market—a “hybrid of people and agents”—competing globally for work (36:08).

• Advice for Hiring Managers

  • “Just for those early employees always index on quality.” (36:36, Foody)
  • Prioritize "talent density," measure hiring results, and beware of “vibes-based” assessments.

Notable Quotes & Moments (with Timestamps)

• On AI surpassing human hiring intuition:
  “I think we'll get to a point where it'll almost be irrational to not listen to the model...where we just trust the model's recommendations on who should be doing a given task or job more than we trust a human's.” – Brendan Foody (02:57)

• On the power law of talent:
  “The power law nature of knowledge work frames the importance of performance prediction…imagine if you can understand the kinds of engineers on an engineering team that are going to perform in the 90th percentile…” – Brendan Foody (03:44)

• On matching inefficiency:
  “There's just so much inefficiency with the way that we do matching, the way we use all of those signals.” – Brendan Foody (08:26)

• On displacement:
  “Displacement in a lot of roles is going to happen very quickly and it's going to be very painful and a large political problem.” – Brendan Foody (09:58)

• On skills that will be most resilient:
  “You just need to be able to...navigate that quickly.” – Brendan Foody (11:53)

• On RFT (Reinforcement Fine Tuning):
  “The reason I'm so optimistic about it taking off is that it's like profoundly data efficient. Right. And it finally makes sense to customize models at the application layer.” – Brendan Foody (32:50)

• On the long-term vision for labor markets:
  “It makes way for a global unified labor market that every candidate applies to and every company hires from...but that include AI agents.” – Brendan Foody (35:21)

• On hiring advice:
  “There's always a trade off between hiring speed and hiring quality and you should just for those early employees always index on quality.” – Brendan Foody (36:36)

Timestamps for Important Segments

• [00:47] – What Merkor does and how AI automates hiring
• [02:57] – AI models outperform humans in evaluation—will become irrational not to use them
• [03:44] – The importance of identifying outliers and power law distribution in talent
• [05:21] – Where AI models are strong/weak in candidate assessment
• [09:58] – Anticipated job displacement and social consequences
• [11:53] – What skills humans should invest in to remain valuable
• [14:37] – Shift from academic benchmarks to evaluating "economically valuable" work
• [19:17] – Data collection as potentially the largest knowledge work job
• [29:14] – AI may soon be better as managers and performance assessors, not just as contributors
• [33:29] – Merkor’s core strategic focuses: talent supply and performance prediction
• [35:21] – Vision for a hybrid global labor marketplace of people and agents
• [36:36] – Advice for hiring at early-stage startups

Conclusion

This episode offers a comprehensive look at the rapidly evolving intersection of AI, hiring, and the nature of work. Brendan Foody provides a data-driven assessment of the opportunities and challenges AI brings to labor markets, pointing toward massive efficiency gains, potentially painful transitions, and a future where both human ingenuity and AI agents compete and collaborate in a global job marketplace. The insights here are essential for anyone thinking about hiring, investing in their own career, or attempting to forecast the future of work in an increasingly automated world.