Hands-On Windows 143: Copilot Vision - All TWiT.tv Shows (Audio)

Summary

Podcast Summary: Hands-On Windows 143: Copilot Vision

Podcast Information:

Title: Hands-On Windows 143: Copilot Vision
Host: Paul Throt (TWiT)
Release Date: June 5, 2025
Description: Leo Laporte brings together some of the most interesting personalities in technology to discuss the most important issues. Fun, relaxed, informative, and always entertaining, TWiT offers the best tech podcasts in the world.

1. Introduction to Copilot Vision

[00:01 - 03:31]

Paul Throt kicks off the episode by introducing Copilot Vision, labeling it as a modern iteration reminiscent of the nostalgic Clippy assistant. He contextualizes Copilot within Microsoft's expanding ecosystem:

"Copilot is getting complex... It's also part of Bing Search now. There's a standalone Microsoft 365 Copilot app, and it's integrated into individual Office apps across Windows, Mac, and mobile platforms."
— Paul Throt [00:45]

Paul highlights the fragmentation of Copilot across various platforms, making it challenging for users to track its features and functionalities.

2. Overview of Copilot's Presence Across Microsoft Platforms

[03:31 - 04:59]

This segment includes promotional content from sponsors, which Paul instructs to skip per user instructions. The sponsors include Red Canary, Electronic Frontier Foundation (EFF), and Center for Internet Security (CIS). These segments provide advertisements and acknowledgments but are not part of the core discussion on Copilot Vision.

3. Demonstration of Copilot Vision in Microsoft Edge

[04:59 - 15:51]

a. Initial Interaction with Copilot Vision

Paul attempts to engage Copilot Vision on his Windows computer but notes its current unavailability on his device:

"This particular computer does not yet have Copilot Vision, but if it did, I would click this and then I could talk to it."
— Paul Throt [06:02]

He then showcases Copilot Vision within Microsoft Edge, where the feature is actively supported:

"Edge has this Copilot pane on the side... and it supports Copilot Vision."
— Paul Throt [06:08]

A brief interaction with Copilot AI ensues:

"Copilot AI: Hey, Paul."
— Copilot AI [06:38]

"Paul Throt: All right, so it comes up, it asks, it says, hi, Hi. Hello copilot. We can talk to it."
— Paul Throt [06:39]

b. Summarizing Text-Based Content

Paul demonstrates Copilot's ability to summarize a written article:

"Summarize the main points of this page."
— Paul Throt [06:45]

Copilot efficiently provides a concise summary of the article's content:

"It's talking about the author's journey in setting up a network attached storage device…" [07:45]

Paul appreciates this targeted summarization, especially when seeking specific information relevant to consumers within a business-focused article.

c. Analyzing Images with Copilot Vision

Transitioning to image recognition, Paul tests Copilot Vision's ability to describe an image:

"Hi, can you tell me what this is a picture of?"
— Paul Throt [07:28]

Copilot responds accurately:

"This image shows a sleek laptop, likely a Microsoft Surface, known for its detachable keyboard and touchscreen capabilities."
— Copilot AI [08:30]

d. Summarizing Videos

Paul explores Copilot's functionality with video content, using a YouTube video by Dave Plummer as an example:

"Can you tell me what the top five points of this video are?"
— Paul Throt [12:12]

Copilot summarizes the video effectively:

"First, the video dives into the origin story of the Windows Start menu…" [12:20]

e. Technical Insight into Copilot's Functionality

Paul delves into the mechanics behind Copilot Vision, explaining that it primarily relies on transcripts for summarizing content across various formats like text, PDFs, and videos.

"It's just doing the same thing. Whether it's this transcript from the video, a web article of whatever kind, a PDF file, it's using the same type of AI to examine the text, provide a summary..."
— Paul Throt [13:30]

f. User Experience and Future Prospects

Paul discusses the potential integration of Copilot Vision into the standalone Copilot app on Windows, anticipating broader availability:

"Once this comes online in the Copilot app on Windows, I would be able to do it from there as well."
— Paul Throt [14:20]

He reflects on the evolving nature of human-AI interactions, noting behavioral shifts as users engage more naturally with AI counterparts:

"It's hard... you find it's kind of strange, like how you fall into this pattern with it where you... apologize for interrupting or you say thank you."
— Paul Throt [14:50]

Paul concludes this segment by emphasizing the versatility and power of Copilot Vision, positioning it as a significant advancement within Microsoft's suite of AI tools.

4. Conclusion and Final Thoughts

[15:51 - 16:26]

The episode wraps up with a promotional advertisement involving T-Mobile and Zoe Saldana, which falls outside the scope of the content-focused summary and is thus omitted.

Key Takeaways:

Copilot Vision represents Microsoft's endeavor to integrate AI assistance seamlessly across its platforms, akin to the nostalgic presence of Clippy.
The feature is currently operational within Microsoft Edge, allowing users to interact via text and voice to summarize and analyze content ranging from articles to images and videos.
Copilot's Integration Challenges: The widespread deployment of Copilot across various Microsoft services creates a fragmented user experience, making it challenging to locate specific features.
User Interaction Dynamics: The naturalization of interactions with AI, such as expressing gratitude or seeking feedback, reflects the deepening relationship between users and AI assistants.
Future Prospects: Anticipated broader integration of Copilot Vision into the Windows app and other platforms promises enhanced accessibility and functionality for users.

Notable Quotes:

"Copilot is getting complex... It's also part of Bing Search now."
— Paul Throt [00:45]
"Edge has this Copilot pane on the side... and it supports Copilot Vision."
— Paul Throt [06:08]
"It's just doing the same thing... using the same type of AI to examine the text, provide a summary."
— Paul Throt [13:30]
"It's hard... you find it's kind of strange, like how you fall into this pattern with it where you... apologize for interrupting or you say thank you."
— Paul Throt [14:50]

This episode of Hands-On Windows offers an insightful exploration into Copilot Vision, shedding light on its current capabilities, underlying technology, and future potential within Microsoft's AI ecosystem. Paul Throt effectively balances technical demonstration with personal reflections, providing listeners with a comprehensive understanding of this emerging feature.

Transcript

Paul Throt (0:01)

Coming up next on Hands on Windows, Clippy is back. Well, not exactly, but there is something called Copilot Vision, which will feel eerily familiar. Podcasts you love from people you trust. This is Twit. Hello, everybody, and welcome back to Hands on Windows. I'm Paul Throt, and this week we're going to take a look at one of many, many new Copilot features, code called Copilot Vision. Sort of the return of Clippy. You'll see what I mean by that soon. Copilot is getting complex. We've talked a lot about Copilot this year. We've talked a lot about AI, all the new features that are coming. But one of the weird things about Copilot is that it's in a lot of places, right? So I tend to focus on it in Windows, which makes sense. It is an app. That app has changed a bunch. But Copilot is also on the web@copilot.Microsoft.com it's part of Bing Search now. It's also Bing searched as part of Copilot. Now. There's a whole mess there. There's a standalone Microsoft 365 copilot app that's in Windows 11, but it's also in the individual Office apps, right. On Windows and the Mac, right. It's on mobile. So there's a standalone Copilot app, Microsoft 365 Copilot app, and then other Microsoft apps that have Copilot capabilities. And so once you get into this weird matrix of Copilot implementations, sometimes it gets hard to know where certain things are, right? So Microsoft introduces these new features. You're like, great, well, where do I get that? It's not always obvious. So I'm going to bring up the Copilot app in Windows to show you just one example. So in Windows, if you go into settings, one of the unique features you'll see here is this phone connection feature. So in Windows only, I can have Copilot give me information that's based on things that are on my phone, right. In addition to things that are on Windows or on the web or wherever else. So that's kind of interesting. There's this voice mode where you have different voices to choose from. This is kind of a minimalist user interface. If I choose a different voice, it doesn't give me a sample. And it also doesn't have other options that are available in other places. So, for example, if you were to bring this up on your phone in the Copilot app, you would have the opportunity not to just hear, try different voices. But you'd hear those voices and you could control the speed of those voices. So you could speed them up or slow it down. Which actually, I have to say is kind of a. Kind of a useful feature. So for some reason that's not in Windows yet. So one feature that's like that, it's brand new, announced recently, and it's just starting to roll out as I record. This is something called Copilot Vision. And this is in some ways Copilot, as I think people think of it when you think of what a copilot is. Something, someone or something in this case that is next to you, helping you as you do your thing. To date, it's been a user interface, so it's a pane or whatever that you see on the side of another app. Maybe in this case, you're interacting it with your voice. And so in addition to typing queries and asking questions, you. You can actually just use your microphone and talk to it. Right. And that, you know, depending on how you do things, that might be better, worse, you know, it might be just a different way to do it. So it's just a different way to look at it.