Podcast Summary

Podcast: Lenny's Podcast: Product | Career | Growth

Episode: Why experts writing AI evals is creating the fastest-growing companies in history
Guest: Brendan Foody, CEO of Mercor
Host: Lenny Rachitsky
Date: September 18, 2025

Episode Overview

This episode dives deep into the explosive growth of Mercor, an AI labor marketplace that connects top talent with leading AI labs and application companies to develop, evaluate, and fine-tune model capabilities. Brendan Foody, the youngest unicorn founder and CEO of Mercor, shares the story behind Mercor’s unprecedented growth and explains why the ability to write and interpret AI "evals" (evaluation rubrics for model performance) is at the heart of the AI economy. The discussion covers the role of human experts in advancing AI, how the labor market is evolving, the future of work, and practical career advice for those seeking to thrive in an AI-driven world.

Key Discussion Points and Insights

1. The New Era of "Evals" in AI (05:51–09:26)

• Definition & Importance:

  • Evals are becoming as critical as the product itself. If "the model is the product, then the eval is the product requirement document" (06:39).
  • Evals set the benchmarks by which models are developed, tested, marketed, and improved.

• Business Opportunity:

  • Mercor capitalized early on the shifting AI labor market, supplying high-caliber professionals (like lawyers, engineers, doctors) to define success for models, as opposed to low-skill data labeling.
  • "We grew from $1M to $400M in revenue run rate in 16 months. Fastest ascent in history." (11:49)

• Evals as Sales Collateral:

  • Evals are becoming not just a research tool but a marketing asset for AI labs and app companies to demonstrate their models' capabilities.

Notable Quote

Brendan Foody (06:39): "If the model is the product, then the eval is the product requirement document. ... And in many ways the barrier to applying agents, the entire economy to automate every workflow is how do we measure success, how do we eval it, how do we and write the PRDs for everything that we want agents to do, which Mercor is obviously a huge part of."

2. The Talent Marketplace Shift: From Crowdsourcing to Expert Sourcing (10:06–14:19)

• Origin Story:

  • Mercor's founders, starting at age 19, automated tech hiring using LLMs, then quickly pivoted to supply AI labs with experts.
  • The market evolved from "crowdsourcing low/medium skill labor" to “highly-skilled, vetted professionals” who define/model expert knowledge and decision-making.

• Concrete Examples:

  • E.g., a lawyer creating a rubric for contract redlining so that AI can learn what expert output looks like.

Notable Quote

Brendan Foody (13:19): "The market is bound by the amount of things where humans can do something that models can't. … What you could do is have a lawyer create a rubric...and that’s really the foundation to measuring what does progress look like for models."

3. The Rise and Scalability of RLHF and AI Evaluation Jobs (15:22–19:06)

• Evolution in AI Training:

  • Shift from supervised fine-tuning to Reinforcement Learning from Human (and now AI) Feedback (RLHF, RLAIF).
  • Humans now "define some sort of success criteria," not just supply input-output pairs.

• Growth & Demand:

  • Mercor and the whole industry see rising demand for domain experts to ensure continuous model improvement.

• Future-of-Work Implications:

  • These AI eval roles represent a new, lasting job class—even as AI automates other work.

Notable Quote

Brendan Foody (19:06): "It’s highly likely that the entire economy will become an RL environment machine building out all of these worlds and contexts... Very few companies and people have talked about this new category of jobs that’s being created and what that’s going to mean..."

4. Skills and Jobs for a Future-Proof Career (20:22–24:20)

• Advice to Students and Early-Career Professionals:

  • Focus on elastic-demand jobs (where more productivity = more demand), e.g., software engineering, product management, research—rather than inelastic roles (like accounting).
  • Learn how to leverage AI tools to 10x your output rather than fear replacement.
  • Practical assessment: Mercor asks candidates to use any AI tool available to build a product in an hour.

• The New Work Mindset:

  • Success will favor those who "lean into abundance," using AI to do more, not those who resist disruption.

Notable Quote

Brendan Foody (25:10): "It's totally spot on. … AI won’t replace you. People that are really good with AI will replace you."

5. The Economics and Pragmatics of AI Expert Work (36:36–38:01)

• Work Setup:

  • Most expert work is part-time, often side gigs for top-tier professionals; projects can also be full-time.
  • Pay rates are highly attractive ($95/hr median; up to $500/hr for exceptional expertise).
  • The top 10% of contributors drive most of the model's improvements, mirroring startup team dynamics.

• Examples:

  • Experts hired range from Harvard Lampoon comedy writers (to make models funnier) to award-winning screenwriters and radiologists.

Notable Quotes

Brendan Foody (37:13): "Our median pay rate in the marketplace is $95 an hour, but it can flex up well up into like $500 an hour based on the depth of someone's expertise."
Brendan Foody (35:02): "We move really fast. … there's also ... in a set of 100 people that we hire, oftentimes the top 10% of people will drive majority of the model improvement."

6. Mercor’s Rocket-Ship Growth and Operating Principles (39:20–46:04)

• Keys to Success:

  • Leading Indicators: Relentless focus on where the market is going, not where it is.
  • Customer Obsession: Word-of-mouth drove growth—no sales or marketing team until far into hyper-scale.
  • Cultural Principles:
    • "Can do" attitude (set huge goals, build around them).
    • Very high hiring standards (patient with early hires).
    • Intensity (output-centric, legendary focused culture).

• Learning from the Market:

  • Mercor’s biggest break was recognizing the pull for high-quality experts from labs; validating the market with big, fast-growing revenue once they adjusted their focus.

Notable Quotes

Brendan Foody (44:54): "If it's difficult to sell, … you're not going to be able to grow a huge business. What you actually need to find is the customer that's surprisingly easy to sell into, where you're going to be able to grow with them."

Brendan Foody (46:04): "The first one is having a can-do attitude...The second thing is really high standards...And then the third...is intensity."

7. AI Progress, Evals, and the Road to AGI (57:15–59:05)

• Evals as Evergreen:

  • No matter how models evolve, there will always be the need for humans to set evals defining "what does success look like" for AI.

• AGI Timelines:

  • Brendan is skeptical about superintelligence arriving soon; expects a decade or more focused on automating knowledge work via ever-better eval-based training.

Notable Quote

Brendan Foody (58:20): "[Superintelligence soon] is a longer road and that's not to diminish from how extraordinary the models are. ... That long road is paved with all of the evals that help to make those capabilities possible."

8. Brendan's Story, Advice, and Lightning Round (55:37–64:54)

• Early Hustle:
  • Shared his middle-school "Donut Dynasty" business as a lesson in initiative: "You can just do things..." (55:51)
• Mercor’s Name: Marketplace in Latin, reflecting ambition to be the global job market.
• Dyslexia: Views it as a strength, helping him bring creative, non-obvious insight to Mercor and management—focus on maximizing strengths.

Notable Quotes

Brendan Foody (64:24): "You can just do stuff...take the leap of faith."

Brendan Foody (65:45): "[Dyslexia] helps me to think a little bit differently, to be more creative and perhaps see the ways that markets are changing that not everyone sees...we focus much more on how we can leverage people's strengths rather than helping to improve weaknesses."

Notable Quotes

• On Evals:
  • "If the model is the product, then the eval is the product requirement document." (06:39)
• On Learning AI Work:
  • "AI won't replace you. People that are really good with AI will replace you." (25:10)
• On Mercor’s Success:
  • "No one in sales and marketing...we focused 100% of company resources on how do we build great products and experiences for our customers." (39:20)
• On Initiative:
  • "You can just do things...the barrier to more companies being built I think is just initiative and taking the steps to build the product or experience that customers want." (55:51)

Timestamps for Important Segments

• The Era of Evals Explained: 05:51–09:26
• Mercor Origin & Market Shift: 10:06–14:19
• Day-to-Day of AI Evaluation Experts: 13:19–17:29
• Which Jobs Will Last in the AI Economy?: 20:22–24:20
• Mercor’s Economics, Pay & Impact: 36:36–38:01
• How Mercor Became the Fastest-Growing Company: 39:20–46:04
• Lessons in Product-Market Fit & Leadership: 44:54–51:44
• On AI Progress & The Limits of Current Models: 57:15–59:25
• Lightning Round (Books, Advice, Personal Story): 63:26–65:45

Final Takeaways & Advice

• Evals will remain crucial for AI progress; domain experts are the new backbone of the AI information economy.
• Elastic-demand roles (software, product, research) and AI literacy offer the best hedge against automation.
• Company-building in "zero-to-one" markets requires both relentless market sensing and uncompromising team standards.
• Personal initiative and the courage to "just do things" trump waiting for perfection or permission.

For hiring opportunities and to learn more, visit mercor.com.

[End of Summary]