No Priors Podcast: "AI is Making Enterprise Search Relevant," with Arvind Jain of Glean

Date: May 15, 2025
Hosts: Elad Gil and Sarah Guo
Guest: Arvind Jain (CEO and Co-founder, Glean)

Overview

This episode centers on how AI—particularly large language models (LLMs) and foundational model advances—have revolutionized enterprise search, transforming a once stagnant field into a source of business value and innovation. The conversation with Arvind Jain draws on his decades-long experience in search (Google, Rubrik, Glean) and explores:

• The paradigm shift from keyword to semantic and generative search
• The technical and organizational challenges of enterprise AI adoption
• Why prior efforts at solving enterprise search failed and what’s now possible
• Glean’s journey using LLMs to help organizations unlock knowledge and productivity
• What it takes to build, deploy, and sell AI-based products to large enterprises

Key Discussion Points & Insights

The Evolution of Search with LLMs

• Search From Keywords to Semantic Understanding:
  • Arvind describes the foundational change made possible by LLMs: rather than matching keywords, search systems can now understand user intent and document meaning deeply to connect questions with relevant answers.

    • "The main thing [LLMs] have done for search is that it has allowed us to really deeply understand a question that a user is asking. And similarly, it allows us to very deeply understand what a document is about and you can actually match people's questions with the right information conceptually." (Arvind, 01:00)

• Glean’s Early Use of Transformers:
  • Glean was ahead of the curve, using early transformer models for “embedding search” before terms like “vector search” or “RAG” existed.

    • "Version one of our product actually already used Transformers for semantic matching. We didn't have these terms. Nobody used to call it vector search. We didn't have rag." (Arvind, 02:34)

  • Glean started by building on BERT, then tailored embeddings to each customer’s business content (03:24).
• Beyond Embeddings—Why Search Is Still Hard:
  • Arvind emphasizes that semantic matching is just one layer. Enterprises need to address data correctness, authority, and relevance—problems not “solved” by LLMs alone.

    • "It's not just enough to say...I'm going to match it with the right information....you gotta actually pick information that's correct today, that is up to date, that has some authority." (Arvind, 03:24–04:00)

Why Enterprise Search Used to Fail—and Why Now

• Technical Barriers in Pre-SaaS Era:
  • Search products previously faltered because it was difficult to access and unify enterprise data spread across on-prem systems (05:23).

    • "It was such a big problem in the pre SaaS world...you just couldn't build a TurnKey product." (Arvind, 05:23)

• How SaaS and APIs Changed the Game:
  • SaaS, APIs, and connectors now make it feasible to aggregate content company-wide, which is foundational for effective search.

    • "SaaS actually allowed you to, to actually build something....you can actually easily go and bring all the enterprise information and data in one place and build this unified search system on top." (Arvind, 05:23)

  • Insider look: A large Glean customer has 1+ billion internal documents, equaling the entire web’s size in 2004 (08:00).

Glean’s Product Evolution and Use Cases

• From Search Box to AI Assistant:
  • The product transformed from Google-like search to ChatGPT-like chat (conversational interface) using company-specific, access-controlled data.

    • "We evolved from being a Google to something that looks more like ChatGPT, more powerful version of ChatGPT inside your company." (Arvind, 10:43)

• Enterprise Apps/Agents:
  • Customers wanted more than generic assistants—they wanted targeted, curated apps (agents) for specific processes (e.g., HR answering only from vetted materials).

    • "HR teams...say that, look, we love green assistant...but sometimes it uses content that's not authorized or blessed by us...Can we create more specific curated experiences function by function for different use cases?” (Arvind, 11:30)

The Reality (and Limits) of LLM Reasoning

• Model Context and Human-Like Limitations:
  • Even with huge context windows, organization and curation of knowledge are critical; simply handing an LLM “everything” doesn’t work.

    • "Models are mimicking human intelligence...Imagine if I...give you...1 million documents...if I give you information in a manner where it's not organized...you're going to have a lot of difficulty reasoning over it. We think about the models the same way." (Arvind, 08:58–10:05)

Access Controls, Security, and Governance

• Enterprise Knowledge Is Mostly Private:
  • Glean must strictly respect and adapt to granular enterprise access controls; data leakage is an existential risk.

    • "You can't...dump all of your internal companies data and knowledge into [a model]...Because if you do that, you're leaking information...Any AI experiences that you build...has to think about security and governance and permissions at a fundamental level." (Arvind, 14:05)

• Building Trust via Permissions:
  • Glean became, by necessity, as much a security product as a search tool—a requirement for enterprise adoption (21:37).

User Behavior, Change Management & AI Education

• Shifting User Habits:
  • Though Glean offers sophisticated capabilities, users default to short, keyword queries (Google-trained behavior). It takes active prompting and education to shift people into richer interactions with AI assistants.

    • "People won't do it. I think everybody has been trained over the last 20 years to actually type in one or two keywords. Like Google has sort of taught us on what search can do...With assistant, people didn't know what to do with it." (Arvind, 16:10)

• Business Case — ROI and Education:
  • Enterprises focus on measurable ROI for AI tools, but Arvind stresses the overlooked importance of upskilling employees and fostering “AI-first” experts.

    • "One thing that often gets overlooked is education because…the world is changing....you want to see people...who are trained and are AI. First they're experts...That has to be like the objective today..." (Arvind, 18:30)

Building and Selling Enterprise AI Products—Go-to-Market Lessons

• Cultural and Organizational Barriers:
  • Unlike Rubrik (which had an established market), enterprise search budgets often did not exist; it required evangelism and educating buyers (19:30).

    • "There was no concept of buying a search product in the enterprise. And everybody thought that, yeah, this is an important problem, but it's not a line item in my business priorities. It's a vitamin, it's a painkiller, people are living without it." (Arvind, 19:30)

  • Paradoxically, good search can alarm customers by revealing governance gaps and sensitive data (“scared of good search”) (20:30).
• Direct Enterprise Sales vs. PLG:
  • Glean wanted to use product-led growth (PLG, “let the product sell itself”), but search natively required indexing company-wide data, favoring a top-down, whole-organization sales approach.

    • "It is by definition a company wide product. Like it's not like, you know, we cannot offer the product to one individual inside a company...we never had that a concept of that we could make it available to one or two or 10 people." (Arvind, 23:22)

  • Ideal approach: pursue both PLG and enterprise sales motions together if possible to maximize reach and timing (24:54).

Building in a "Bad Market"—Advice for Founders

• Disregarding Priors When You See Real Pain:
  • Arvind’s advice: It’s often better to trust your direct experience of a real, widespread problem than to over-weight the failures of past companies.

    • "Sometimes a more simpler approach is helpful which is, well, there's a problem. You talk to people they have and they feel this pain which clearly means that nobody is actually yet solving that because the pain exists...just do it. Things will just get figured out over time." (Arvind, 25:46)

  • He found conviction in the universality of the problem, even as a “graveyard” market.

The Road Ahead: AI as Personal Teams and Work Transformation

• Vision—Everyone Gets a “10x Team”:
  • The future of work: individualized, AI-powered assistants, coaches, and teams that know your context intimately and proactively handle most tasks.

    • "Each one of us is going to have this amazing team of...assistants, coworkers, coaches, that are truly personal to you...you're always surrounded by that team...It proactively helps you, does 90% of your work for you, and also...help you get better." (Arvind, 29:00)

  • Glean's focus remains on perfecting the assistant and agent products, rather than diversifying (27:51).

Notable Quotes & Memorable Moments

• On Why Embeddings Alone Aren’t Enough:

  "Search as a technique...there's a lot of focus on embeddings and vector search over the last few years. But that's actually only one part of building a good search system."
  — Arvind, 03:24

• On SaaS as the Key Unlock:

  "I felt that the biggest problem was actually solved, which was that I could actually easily go and bring all the enterprise information and data in one place and build this unified search system on top."
  — Arvind, 05:23

• On User Education and Adoption:

  "I always felt that we're building such an intuitive product....But we realized that as we added more and more...the ability for you to actually ask a really long question...people won't do it...AI is actually very unintuitive for most people."
  — Arvind, 16:10

• On the Importance of Security and Governance:

  "Any access to data that's going to happen through our platform is going to actually match...the users have to be signed in and we will actually only let them use information that they have permissions for."
  — Arvind, 14:05

• On Overcoming Doubts and Market Skepticism:

  "There are always doubts...the more you look at priors, the more you're going to actually likely…kill your own idea....if...they feel this pain...nobody is actually yet solving that...just do it."
  — Arvind, 25:30

• On the Future of Work and AI Assistants:

  "That's the world that we want to be living in...you're always surrounded by that [AI-powered] team...that's going to make us all a 10 XL."
  — Arvind, 29:00

Timestamps for Major Segments

• LLMs Transform Search Paradigm: 01:00
• Glean’s Early Adoption of Transformers: 02:34–03:24
• Why Old Approaches to Enterprise Search Failed: 05:18–06:41
• The SaaS & API Revolution Enables New Search: 06:41–08:00
• Scaling and Data Explosion in Enterprises: 08:00
• Semantic Search Alone Isn’t Sufficient: 08:43
• Conversation-style Interfaces and AI Apps: 10:43–13:44
• Access Controls and Security Challenges: 14:05–15:36
• User Behaviors & Need for AI Education: 16:10–18:30
• Enterprise Sales vs PLG Debate: 23:22–24:54
• Advice for Founders in “Bad Markets”: 25:30–27:11
• The Vision of AI-as-a-Team: 29:00

Tone and Remarks

The atmosphere is conversational, candid, and packed with practical wisdom—balancing technical depth, product lessons, and entrepreneurial philosophy. Arvind is reflective about both the difficulties and opportunities of enterprise AI, and the hosts probe for actionable takeaways, making this episode highly useful for founders, technical leaders, and anyone interested in the future of work.

For further information, visit nopriors.com for transcripts and more episodes.