Podcast Summary: The AI Daily Brief – "How AI Companies Are Using AI"

Host: Nathaniel Whittemore (NLW)
Date: July 3, 2025

Overview

In this episode, Nathaniel Whittemore (NLW) takes a deep dive into a new report by Iconic, focusing on how companies that actually build AI are leveraging artificial intelligence internally. The discussion moves beyond typical enterprise AI adoption and provides insights directly from “AI builders,” highlighting the technologies, challenges, and strategic trends unique to this critical segment of the industry. The episode also compares these insights with broader enterprise trends to uncover overlaps and differences in AI practice and adoption.

Key Discussion Points & Insights

Types of AI Companies: A Spectrum of “Builders”

• Categories Identified (04:00):
  • AI-Enabled Companies (adding AI to existing products): 31% of respondents—e.g., Atlassian integrating AI into work management.
  • AI-Enabled (new, non-core AI products): 37%—e.g., Salesforce's Agent Force.
  • AI-Native Companies (core product is AI): 32%—e.g., 11 Labs.

“Despite these all being AI builders, there’s actually quite a range.” (04:30)

Development and Deployment Status

• AI-Native companies progress faster; only 10% stuck in beta compared to 34% among non-native.
• Scaling: 47% of AI-Native are scaling AI products versus just 13% of AI-Enabled.

What Are AI Companies Actually Building? (07:15)

• Key Focus Areas:
  • Agentic workflows (agents, automation)
  • Vertical and horizontal AI applications
  • AI platforms, infrastructure, and core models
• Agent Development:
  • 62% of AI-enabled and 79% of AI-native companies are developing agents.

Model Usage and Selection (09:10)

• Average models used: 2.8 per company.
• Top Providers: OpenAI (leader), followed by Anthropic, Google, Meta, Mistral, Deepseek, Cohere.
• Model Selection Criteria:
  • Accuracy: Top consideration (74%).
  • Cost: Rapidly rising (57% list as top-three, jumped from previous year where it ranked lowest).
  • Open Source/Vendor Lock-In: Not a priority (9% and 6%, respectively).

“To the extent that your company is laboring over a decision around which model to use, we may be at a stage where trying different models for different purposes may be the play.” (10:30)

Key Challenges in Model Deployment (12:00)

• Top Issues:
  • Hallucinations
  • Explainability & trust
  • Improving ROI
  • Compute cost
  • Use case discovery: 25% cite as a challenge.

The Status of AI Agents in Production (14:00)

• High Growth Companies:
  • 47% have agents in production.
  • 42% piloting or using agents internally.
• Other Companies:
  • 32% agents in deployment; another 32% piloting.

“Agents are here, baby.” (14:15)

AI Leadership & Talent Acquisition (16:10)

• Dedicated AI leadership becomes standard at $100M+ revenue.
• Hiring Focus: AI engineers, data scientists, prompt engineers, AI product managers, design specialists.
• Hiring struggle: 46% say not hiring fast enough; 60% of those cite lack of qualified candidates.
• Biggest cost center: Talent—36% of budget (vs. 12% for model training, 10% inference).

Cost Controls and Open Source Interest (18:45)

• Costliest line item to control: API usage fees (70% cite as top difficulty).
• Strategies: Despite lack of initial open source preference, 41% consider open source models to manage costs.

Measuring & Realizing AI ROI (20:25)

• Most-tracked ROI Metric: Productivity gains (75% monitor this).
• Other metrics: 51% monitor cost savings; just 20% track revenue increase.
• Contrast with enterprises: Enterprises more focused on transformation & revenue growth, while AI builders focus on operational effectiveness.

Internal AI Productivity Budgets (22:55)

• Budgets are doubling for 2025.
• Shift in budget allocation: Funding is moving from “innovation” budgets (47% → 23% YoY) to core R&D/business budgets, signaling mainstream adoption.

Internal AI Use Cases (27:42)

• Most Significant Challenge: Finding the right use cases (46% cite as top issue).
• Correlation: More employee adoption → more use cases (7.1 avg if >50% of team using AI).
• Top use cases:
  • Coding assistance (77%)
  • Content generation (65%)
  • Documentation/knowledge retrieval (57%)
  • Product & design (56%)
  • Sales productivity (45%)
  • Customer engagement (42%—lower among young companies)

AI Impact & Coding Assistance (32:00)

• Productivity gains: 15–30% common across use cases.
• Coding assistance: Most impactful; 33% of code in high-growth companies now AI-generated (27% for others).

“This is just so clearly the biggest breakout use case so far, and one that’s having a huge impact right now.” (33:45)

Business Models & Pricing Experiments (35:10)

• Current models:
  • Hybrid (38%)
  • Subscription (36%)
  • Usage-based (19%)
  • Outcome-based (6%)
• 37% plan to change their pricing model in the next year—expect more consumption and outcome-based pricing.

The AI Builder Tech Stack (36:30)

• Comprehensive stack: From model training, LLM development, monitoring, inference optimization, data, evaluation, DevOps/ML Ops, to product design.
• Tools Called Out:
  • Coding assistance: GitHub Copilot (75%), Cursor (50%).
  • Model evaluation: No clear leader; 20% don’t know what they use, 25% lack a tool—huge opportunity area.

“Model evaluation ... seems almost destined to grow in focus in the coming months.” (38:15)

Incumbents vs. Startups in Internal Productivity (39:30)

• Incumbents win at integration:
  • Sales: Most use Salesforce’s AI.
  • Marketing: Canva for branded visual generation.
  • Customer engagement: Zendesk and Salesforce for AI features.
• More innovation in new problem spaces: e.g., documentation, knowledge retrieval.

“The incumbents really do have an advantage… even among the startups themselves.” (40:00)

Notable Quotes & Moments

• On the Transformation of AI Adoption:

  “We are firmly out of the pilot and experimentation stage. We are experimenting with new business models. Spend and budgets are increasing. Talent really matters.” (02:10)

• On Cost Becoming a Bigger Factor:

  “When you're actually scaling a product that's going to have tons of usage, boy does cost make a big difference.” (12:50)

• On Use Case Discovery:

  “One really interesting ... was a quarter of the respondents had listed finding the right use cases as a top three challenge, which is interesting considering that these are companies who are building AI models.” (13:15)

• On Human Talent:

  “The biggest cost center for companies is around talent. ... A full 36% of their AI budget is allocated to salaries, hiring and upskilling.” (17:45)

• On Coding Assistance:

  “For those high growth companies, an average of 33% of their total code is currently being written by AI. ... This is just so clearly the biggest breakout use case so far.” (33:00)

• On Incumbents’ Advantage:

  “It really does bring up just how challenging it’s going to be for all these vertical AI companies who have to compete against these legacy platforms ... even among the startups themselves.” (41:30)

• On Industry Progress and Pain:

  "It's a confirmation of both how fast the industry is moving, but also that we're all in this together and that even for the companies who are building the technology, much of these transformations are incredibly, incredibly difficult." (42:00)

Timestamps of Important Segments

• Types of AI Companies: 04:00–06:30
• Product Status and Scaling: 06:30–07:15
• AI Model Usage and Selection: 09:10–11:30
• Model Deployment Challenges: 12:00–13:15
• AI Agents Adoption: 14:00–15:15
• AI Leadership & Talent: 16:10–18:10
• Cost Controls and Open Source Trends: 18:45–20:20
• Tracking and Measuring ROI: 20:25–22:30
• Internal AI Productivity Budgets: 22:55–25:45
• Internal AI Use Cases & Adoption: 27:42–31:30
• Coding Assistance and Productivity Impact: 32:00–34:00
• Business Model Experimentation: 35:10–36:00
• AI Builder Tech Stack Overview: 36:30–38:30
• Incumbents vs. Startups (Integration): 39:30–41:30

Conclusion

This episode is a must-listen for anyone interested in how AI companies are deploying AI internally, not just as vendors but as internal users. NLW’s breakdown of the Iconic report sheds light on the fast-moving practices, sticking points, and culture of experimentation inside the world’s most advanced AI builders. The episode confirms both the velocity of innovation and the persistent, universal challenges—especially around talent and use case discovery. The overarching message? Whether you're a scrappy startup or a Fortune 500, deploying AI at scale remains as challenging as it is essential.