Hands-On Windows 158: Semantic Search

Host: Paul Thurrott
Podcast: All TWiT.tv Shows (Audio)
Date: September 18, 2025

Overview

In this episode, Paul Thurrott delves into the new semantic search capabilities in Windows 11, focusing on how artificial intelligence (AI) and on-device small language models are transforming the way users find files and information. Paul shares his experiences, demonstrates the updated search interface, discusses its current hardware limitations, and offers practical insights into how semantic search outperforms the traditional index-based search.

Key Discussion Points & Insights

1. The History and Pain of Windows Search

• Paul reminisces about past attempts at improving Windows search; features like Longhorn and WinFS never solved the fundamental issues.
• The traditional index-based search, built into the Start menu and File Explorer, is often clunky or inaccurate—even after decades of development.
  • Quote:
    "If you have been around Windows for a while like I have, then you probably remember things like Longhorn and WinFS and database spec file systems and how we were going to solve all those file search problems and then didn't. Yeah, this never happened." (02:46)

2. What is Semantic Search?

• Semantic search leverages local, on-device AI (small language models) available exclusively on Copilot+ PCs for now.
• Unlike keyword-based search, semantic search interprets queries in natural language, returning results based on context rather than just file names or types.
• Paul's demonstration explores the familiar search UI, with a new sparkly purple-blue icon indicating the enhanced capability.
  • Quote:
    "It's using its natural language capabilities to understand what it is you're looking for." (05:56)

3. Demonstration: How Semantic Search Works

• Paul illustrates using semantic search by seeking out documents related to "Longhorn" (a previous Windows code name and frequent writing subject for Paul).
• He notes improved relevance even with files stored remotely (e.g., old articles in OneDrive archives), and the AI can find content not locally synced.
• Cloud support is currently limited to OneDrive, but Paul finds it works with other services like Synology Drive via open APIs.
  • Memorable Moment:
    Paul searches for "Longhorn" and gets results for Longhorn Steakhouse, illustrating both the strengths and quirks of the system.
    "So you're getting the best match is the Longhorn Steakhouse, which is absolutely not what I was looking for." (09:20)

4. Semantic Search in the Settings App

• The Settings app in Windows 11's upcoming releases (version 1125 H2 and 24 H2) will support semantic/natural language queries.
• Example: Searching "my mouse pointer is too small" yields helpful, actionable suggestions.
  • Quote:
    "It's using its natural language capabilities to understand what it is you're looking for. And it's even providing a way to kind of do it right there in line. Right. Which is nice." (05:59)

5. Indexing Options and Performance

• Enhanced indexing (vs Classic) is recommended by Paul for broader and faster search capabilities, including across multiple cloud and network drives.
• The semantic search feature still relies on the legacy file index but processes it through AI to generate more meaningful results.
• The modern UI for indexing is in the Settings app under Privacy & Security > Search.
  • Quote:
    "For best results I found...it's set on Classic. By default I turn this on to Enhanced and it's basically the entire PC." (13:55)

6. Limitations & The Road Ahead

• Limitation: Requires a Copilot+ PC with an appropriate neural processing unit (NPU); currently, less than 10% of users have such devices.
• Paul is optimistic this feature will be expanded to non-Copilot+ devices.
  • Quote:
    "So the problem with this to me is only that you have to have a Copilot plus PC to get this. But like I said, I think it's going to come to everybody." (12:09)

7. The "AI Moment" for Search

• Paul describes the move to semantic search as the long-awaited breakthrough for Windows search, achieved thanks to advancements in local AI.
• Emphasizes how AI allows searches to be based on understanding document content and context, not just metadata.
  • Quote:
    "Instead of just searching against say the file name or if it's known file type, like the file contents, it actually can search against AI's understanding of what is in those things." (17:26)

Notable Quotes

• On the evolution (or lack) of search in Windows:
  "We're still using the same index based search functionality that we've always had...if anything they've made it worse." (02:57)

• On semantic search's potential:
  "It's something that we have been struggling with in Windows for so many years. And this is, I think this is going to kind of solve it." (15:53)

• On real-world improvement:
  "In my experience, it actually works faster and better. So don't often say that about AI, but that's the way it works today." (18:12)

Timestamps for Important Segments

• 02:46 – Opening: The long history of poor search in Windows
• 04:35 – Introduction of Copilot+ PCs and semantic search
• 05:56 – Natural language search in the Settings app explained
• 09:20 – Demo: Searching for "Longhorn" and unexpected results
• 12:09 – Limitation: Copilot+ PC requirement
• 13:55 – Enhanced indexing settings recommendation
• 17:26 – How AI fundamentally changes search relevance
• 18:12 – Host’s verdict: Faster, better, and the long-awaited solution

Conclusion

Paul Thurrott’s deep dive into semantic search showcases a genuine leap forward for file discovery on Windows. While hardware limitations mean most users can’t access it yet, the demonstration highlights how local AI is finally delivering on Microsoft’s decades-old promise of smarter, more useful search. Paul’s practical examples, technical walkthroughs, and candid assessment equip listeners with a clear understanding of where Windows search is heading and why it matters.

“If you do have a Copilot plus PC, definitely check this stuff out. Like I said, it just, to me, it makes finding things that are in your file system or up in the cloud, just a lot faster.” —Paul Thurrott (19:18)