Embracing Digital Transformation

Episode: Small Language Models: The Public Gen AI Killer?

Host: Dr. Darren Pulsipher

Guest: Lynn Kampf, Intel Corporation

Date: October 30, 2025

Episode Overview

This episode dives into a major trend shaping enterprise AI: the rise of Small Language Models (SLMs) and their potential to outpace public, generic generative AI in practical, scalable business applications—especially for the public sector. Dr. Darren Pulsipher and returning guest Lynn Kampf (Intel Corporation) break down the hype, challenges, and actionable ways for organizations to benefit from SLMs, focusing on efficiency, risk reduction, and real-world use cases.

Key Discussion Points & Insights

1. Enterprise AI Benchmarks and Token Confusion

• Token Metrics Explained [(01:35–05:03)]

  • Mainstream AI benchmarks (like tokens per second) are often misleading for business users.
  • Lynn Kampf:

    “Tokens are a Greek supply side metric because it's very similar to megahertz. How quickly can you get bits out? ... But we're not really translating what that means to humans.” (02:12)

  • High token throughput, like speed in a race car, isn’t always needed for human-facing use cases.
  • Critical question: Does faster output provide real business value (e.g., in fraud detection) or just higher cost?

• Analogy:

  • “Ferraris are amazing on racetracks ... Ferraris that are driving through a school zone to get groceries, not very good. Think of human consumption as the school zone ... we don't operate at the speed of light, the machines do.” [(03:30)]

2. Cost, Infrastructure & Efficiency: Why SLMs Matter

• Hidden Costs of High-Volume AI

  • More tokens per second = higher compute costs and potentially specialized, expensive hardware [(05:08)].
  • Most enterprise IT is only ~40% utilized, with the rest essentially “sitting idle for backup.”
  • SLMs let organizations leverage existing infrastructure (CPU/GPU), minimizing disruption and cost.

• Tailored, Vertical Models Add Value

  • SLMs/vertical models are designed for specific industries, offering efficiency where general models (e.g., GPT-4) offer little extra value:

    “If your business is not needing to write haiku, input Shakespeare and create video images for fun, then you may not get value out of the biggest punches.” (Lynn Kampf, 06:48)

• Strategic Start:

  • Begin with small, impactful use cases (summarization, policy analysis, data sync) and scale as demand grows.

3. Pragmatic AI Adoption in the Enterprise

• Where Most Fail [(09:08–10:22)]

  • Refers to a recent MIT report: “95% of Gen AI business PoCs are failures.”

  • Cause: Rushing to use AI “for the sake of AI,” poor data preparation, or not integrating with existing business data.

  • Quote:

    “Most enterprises ... if it's a mature stable tech stack and it's got disaster recovery, you leave that sucker alone. You're not just going to go add new technology because it's new.” (Lynn Kampf, 09:26)

• Avoiding Data Silos

  • Implementing separate AI systems can create data silos and duplicative costs.
  • Best Practice:

    “Prototype small and tight. Take advantage of the extra capacity you have and then really judge whether new hardware is necessary based on—will I get [more] if I'm running faster, or will I decrease my liability?” (10:53–12:41)

4. Concrete SLM Use Cases

• Summarization & Knowledge Management [(12:56–14:56)]

  • Example: Internal tools like “beta”—a searchable database of support calls and solutions.

    “Having something very quick that summarizes—is somebody else seeing this, what is the possible part?” (13:08)

  • Call centers benefit from SLM-empowered search and summarization—driving efficiency and reducing operational costs.

• Policy Analysis & Expense Auditing

  • Internal chatbots surface travel or HR policies instantly.
  • SLMs audit expense reports by finding “out of sorts” cases for human review.

    “It just helps the human not have to do as many actions.” (15:28)

• Retaining the Human Element:

  • SLMs and vertical models support decision-making but don't replace judgment.

    “AI is not going to be [a] good judge. AI is going to help very quickly collect information ... so that you can use judgment and critical thinking.” (16:07)

5. The Impact on Workforce & Skills

• Upskilling for AI-Augmented Roles [(16:43–19:43)]

  • Humans will focus less on “glue” work (manual data movement/interpretation) and more on judgment and recommendations.
  • Dr. Pulsipher:

    “We as humans have to train ourselves more on critical thinking, art, mitigations. ... We've been relying too much on ... gathering all the data, I don't know what I'm supposed to show.” (18:10–18:44)

• The New AI-Partnered Paradigm

  • Employees become “AI augmented,” not replaced—maximizing business intelligence rather than repetitive clerical work.
  • Kampf’s Quip:

    “Blind AI is do stupid stuff ... at the speed of light. So you just need that human [to] accelerate your judgment.” (19:43)

6. Architecture and Hybrid AI Environments

• Hybrid Is the Future [(20:44–22:18)]

  • SLMs and vertical models will run across cloud and on-prem, requiring resilient, adaptable architectures.
  • Avoid “flat lift and shift” mistakes from early cloud adoption; apply lessons learned for cost and resilience.

    “If I architect inappropriately, then I'm not affected by outages. I'm not affected by data breaches.” (21:29)

• Owning Your Output

  • Using public chatbots may result in losing copyright—vertical, context-specific models keep knowledge in-house.

Notable Quotes & Memorable Moments

• Ferrari Analogy (Tokens vs. Business Value):

  “Ferraris are amazing on racetracks ... but not very good in a school zone ... we don't operate at the speed of light, the machines do.” (Lynn Kampf, 03:30)

• On Failed GenAI Projects:

  “95% of Gen AI business PoCs are failures.” (Dr. Pulsipher, referencing MIT, 09:08)

• On Human Judgment:

  “AI is not going to be [a] good judge. AI is going to help... get information so you can use judgment and critical thinking.” (Lynn Kampf, 16:07)

• Human Upskilling:

  “We as humans have to train ourselves more on critical thinking, art, mitigations.” (Dr. Pulsipher, 18:10)

• AI-Augmented, Not AI-Replaced:

  “AI augmented, right? Not AI replaced. If you value your employees and your company then you want more intelligence at the table.” (Dr. Pulsipher, 19:17)

Key Timestamps

| Time | Segment | |------------|----------------------------------------------------| | 01:35–05:03| Token confusion & performance vs. value | | 06:22–07:38| Using current infrastructure with SLMs | | 09:08–10:22| Why most GenAI business PoCs fail (MIT study) | | 12:56–14:56| Use case: Summarization and internal knowledge | | 15:24–16:39| Use case: Policy chatbots, expense audit | | 16:43–19:43| Skills for an AI-augmented workforce | | 20:44–22:18| Hybrid architecture for SLMs and vertical models |

Summary Takeaways

• SLMs are a pragmatic, cost-effective way for enterprises (especially in the public sector) to leverage GenAI without the cost and risks of massive public models.
• Start small, prototype, and align with concrete business needs—avoid creating new silos and focus on augmenting (not replacing) human intelligence.
• Upskill your workforce for the next era: judgment, synthesis, and critical thinking are more relevant than ever.
• Hybrid, resilient architectures are vital—carry forward lessons from cloud migration into enterprise AI adoption for security, copyright, and organizational control.

“The sweet spot is to be strategic: let the tactics run through automation ... adopt what’s appropriate, and you’ll make smarter decisions and elevate your business.”
— Dr. Darren Pulsipher (20:12)

For further resources and deeper dives, visit embracingdigital.org.