The MAD Podcast with Matt Turck

Episode: AI That Ends Busy Work — Hebbia CEO on “Agent Employees”

Guest: George Sivulka, Founder & CEO of Hebbia
Date: May 29, 2025
Host: Matt Turck

Main Theme Overview

This episode features an in-depth conversation with George Sivulka, founder and CEO of Hebbia, an AI platform designed to automate complex knowledge work for high-value professionals. The discussion revolves around the proliferation of AI “agent employees,” the future of workplace automation, and the evolution of AI-driven organizational design. Sivulka shares Hebbia’s journey from a niche financial tool to a generalist platform, delivering insights on product innovation, technical architecture, organizational change, and what it takes to build and scale in the competitive AI landscape.

Key Discussion Points & Insights

1. Hebbia's Mission, Origin Story, and Growth

• Mission Evolution:
  • From simply “keeping smart people from doing stupid tasks” to “putting capable AI in the hands of a billion people” ([03:05]).
  • Focus on building broadly capable, horizontal AI agents for knowledge work, moving beyond vertical solutions.
• Founding Story:
  • Sivulka left a fully funded Stanford PhD after witnessing GPT-3’s capabilities, calling it a "meta-learner" that pushed him to apply AI practically ([04:52]).
  • Youngest in his program, he walked away from millions in research grants without knowing if he could build a successful company ([06:21]).
• Growth & Adoption:
  • Team scaled from NYC to multinational (San Francisco, London), aiming for 300-400 employees by year-end ([07:03]).
  • Platform processes 4-5 billion pages annually (compared to 100 million the prior year), used by 40-50% of the world’s top asset managers and major law firms ([07:03]).
• Notable Quote:
  • “With Hebbia, you can give it these complex tasks and it really turns through vast quantities of data and does work the way you work. It’s much more of an agent than a chatbot.” — George ([07:03])

2. Agent Employees: Redefining Organizational Structure

• Defining Agent Employees:
  • AI agents are conceived as nodes in an organization’s chart, similar to human employees—complete with their own email, Slack, processes, and the need for management ([09:58]).
  • The decoupling of labor from personhood, echoing remote and hybrid work’s revolution in labor location ([09:58]).
• Future of Work:
  • Prediction of hybrid organizations made up of both AI and human workers, with some fully AI and some fully human ([09:58]).
  • Effective “AI managers” are individuals skilled at prompting and guiding agent employees ([12:54]).
• Notable Quote:
  • “People that are really good at prompting, really good at defining a process, be the best managers… everyone will be prompting and prompting is managing and it will all blur pretty soon.” — George ([12:54])

3. Generalist vs. Vertical AI

• Horizontal Power:
  • Rejects verticalized AI applications, arguing that true expert users want generalizable, customizable tools ([21:01]).
  • Best professionals synthesize knowledge from diverse, unrelated fields—AI should reflect that ([21:01]).
• Evolution Beyond Chatbots:
  • Chatbots likened to calculators (good for single responses); real productivity lies in spreadsheet-like, agent-driven platforms ([24:08]).
  • Hebbia’s “Matrix” is described as the new Microsoft Office, with applications that mirror Word, Excel, etc. ([25:43]).

4. Technical Deep Dive: Architecture, RAG, and Scaling

• Data Ingestion & Indexing:
  • Heavy preprocessing of documents to build rich schema; agents pre-do as much work as possible before dealing with user queries ([26:27]).
• Beyond RAG to ISD:
  • RAG (Retrieval Augmented Generation) is described as outdated for their hardest use cases ([28:53]).
  • Hebbia developed “ISD”—leverage preprocessing and recursive, agent-driven document decomposition for richer, more accurate answers ([28:53]).
  • Still have world-leading re-ranker tech, but “search isn’t what’s important” in agentic workflows ([30:28]).
• Model Layer and Scaling:
  • Use multiple models from OpenAI, Anthropic, and Amazon ([31:48]).
  • “Maximizer” system routes billions of LLM calls efficiently, likened to an air-traffic controller ([32:28]).
  • Information-theoretic maximization for rate limits and compute utilization.

5. Context Windows, Multimodality, and Inference Scaling

• Extending Context Windows:
  • The key challenge in AI is to efficiently extend the “context window” so AI can reason over vast, diverse data ([18:01]).
  • Techniques involve decomposing, compressing, and iteratively updating context ([19:49]).
• Inference-Time Scaling:
  • Hebbia pioneered the idea that increasing the number of inference (runtime) calls boosts accuracy in complex tasks ([15:43]).
  • “Scaling at inference” is a major research direction as training scaling laws plateau ([15:43], [38:02]).

6. Minimizing Hallucinations & Reliability

• Hallucinations Outdated?
  • Sivulka downplays current concern, asserting AI models have surpassed human reliability for most tasks and errors now stem from incomplete context, not LLM hallucination ([34:26]).
  • "Intelligence will become too cheap to meter," so expensive, redundant reasoning passes are justified ([34:26]).

7. Research & Industry Perspective

• Independent Research Focus:
  • Deliberate avoidance of Silicon Valley groupthink; Hebbia’s research lab emphasizes context, retrieval, and AGI-oriented interfaces rather than chasing the latest industry trend ([36:11]).
• Cautious on 'Alpha' in AI:
  • Sivulka suggests AI research is nearing a plateau; future robustness and cost reduction are likely, but transformative leaps are slowing ([38:20]).
• Notable Quote:
  • “I wouldn’t start a company in AI right now... the alpha is gone.” — George ([38:20])

8. Go-To-Market & Sales Innovation

• Sales Playbook Differences:
  • Traditional SaaS sales methodologies (like “medic/medpic”) are less effective for AI, which is often sold on FOMO, value creation, and opportunity cost rather than problem-pain ([40:08]).
  • Sales/CS team at Hebbia is heavier on consulting/domain expertise ([40:08]).
• ROI Calculation:
  • Value cases span cost savings (eliminating tasks, third-party spend) but also emphasize hard-to-quantify upside: enabling new analyses, finding market/information edges ([42:56]).

9. Cultural and Leadership Notes

• Leading as a Young Founder:
  • Sivulka credits "naiveté" as a superpower—unaware of the magnitude of the challenge, young founders take on hard problems ([46:23]).
  • Recognizes the importance of building and learning from a complementary team of veterans.

Memorable & Notable Quotes

| Timestamp | Quote | Speaker | |---|---|---| | [00:00] | “You’ll actually have hybrid AI and human employees working alongside each other. People that are really good at prompting be the best managers. Intelligence will become too cheap to meter.” | George Sivulka | | [03:05] | “...The goal of Hebbia was never to just stop at really highly paid knowledge professionals... our vision and mission have solidified... into building capable AI platform for a billion people.” | George Sivulka | | [04:52] | “This thing is a meta learner. It has beaten me to the research punch that I was working on.” | George Sivulka | | [07:03] | “With Hebbia, you can give it these complex tasks and it really turns through vast quantities of data and does work the way you work. It’s much more of an agent than a chatbot.” | George Sivulka | | [09:58] | “You’re going to start to have fully human organizations, actually fully AI organizations like the one person billion dollar startup… and this will be the most common thing: hybrid AI and human employees, agent and human employees working alongside each other.” | George Sivulka | | [12:54] | “Everyone will be prompting and prompting is managing and it will all blur pretty soon.” | George Sivulka | | [15:43] | “There's a lot of really interesting research direction in scaling laws for scaling during inference...” | George Sivulka | | [21:01] | “One of the things that is a massive fallacy in AI applications today is verticalization as paramount... generalization will beat specialization every single time.” | George Sivulka | | [24:08] | “Chatbots are in my eyes like the TI84… like a calculator. Nobody does their taxes in a calculator.” | George Sivulka | | [31:48] | “We believe the model layer will become commoditized… whatever models you want to use…” | George Sivulka | | [32:28] | “We liken [Maximizer] to an air traffic controller... information theoretic, maximum utilization of any rate limits.” | George Sivulka | | [34:26] | “I think [hallucinations are] old news... they’re way better than any human... intelligence will become too cheap to meter.” | George Sivulka | | [38:20] | “I wouldn’t start a company in AI right now... the alpha is gone.” | George Sivulka | | [42:56] | “A lot of what really is driving value... is the idea of, hey, you can actually make way more money with AI... it’s part of the reason people buy.” | George Sivulka | | [46:23] | “I think naiveté is a superpower because it’s incredibly hard and a terrible existence to be a founder. Everyone always says, like, oh, I wouldn’t do it if I knew what it was like. And I think that’s actually true.” | George Sivulka |

Important Timestamps by Topic

• Intro, Hebbia’s Origin, and Mission: [00:00] – [07:03]
• Agent Employees & Org Structure: [09:52] – [13:44]
• Vision of 2030 Organization: [12:36]
• Research Landscape & Inference Scaling: [15:43] – [19:49]
• Matrix Product & Expert Interfaces: [19:49] – [25:43]
• Data Ingestion, Indexing, RAG to ISD: [26:09] – [30:28]
• Model Layer & Technical Scaling: [31:48] – [34:10]
• Hallucinations & Reliability: [34:10] – [34:26]
• Research Culture: [36:11] – [38:20]
• Go-To-Market & Sales Team: [39:57] – [42:04]
• ROI & Value Cases: [42:04] – [43:48]
• Workforce Impact & History: [44:05]
• Personal Leadership Notes: [46:23]
• Vision for Next 2-3 Years: [47:16]

Tone & Language

• Open, Pragmatic, and Deeply Technical:
  The discussion blends technical rigor with practical business insight, often colored by Sivulka’s direct, sometimes irreverently honest, assessments of the industry (“the alpha is gone,” “chatbots are calculators,” “intelligence will become too cheap to meter”).
• Confident but Analytical:
  Sivulka offers both optimism for AI’s impact and realism about industry hype cycles and research plateaus.

Conclusion

The episode offers a comprehensive look at how AI “agent employees” and generalist AI platforms are set to reshape not just repetitive work, but entire organizational paradigms. Through Hebbia’s story, Sivulka argues for horizontal, agentic AI—rejecting the chatbot status quo and verticalized solutions—in favor of infrastructure that enables expertise, customization, and massive scalability. For founders, operators, and anyone interested in AI’s workplace impact, this conversation is rich with technical insight, strategic perspective, and pragmatic predictions about where the industry is headed next.