Hands-On Mac 166: Apple's New AI Clean Up Tool - All TWiT.tv Shows (Audio)

Summary

Podcast Summary: Hands-On Mac 166 – Apple's New AI Clean Up Tool

Release Date: January 23, 2025
Host: Micah Sargent
Podcast: All TWiT.tv Shows (Audio)

Introduction

In episode 166 of Hands-On Mac, hosted by Micah Sargent, the focus is on Apple's latest artificial intelligence feature introduced in macOS Sequoia: the Cleanup tool. This innovative feature is designed to help users effortlessly remove unwanted distractions from their photos, enhancing the overall visual appeal with minimal effort.

Overview of Apple's Cleanup Tool

Micah begins by introducing the Cleanup feature within the Photos app, emphasizing its integration with Apple Intelligence. The tool leverages on-device machine learning models to identify and remove selected elements from photographs, ensuring user privacy and security by processing data locally.

"Cleanup is a feature for macOS Sequoia and for devices that support Apple Intelligence that will attempt to remove the toy." [01:48]

Demonstration and Examples

Removing Distractions from a Pet Photo

Micah demonstrates the Cleanup tool using a photo of his dog, Henry. The image initially contains a distracting toy in the background. Instead of cropping the photo, Micah opts to use Cleanup to remove the toy, aiming to keep the focus solely on Henry.

Selection Process:
- Navigates to the Edit menu in the Photos app.
- Chooses the Cleanup option, which initiates a download of the necessary on-device model.
- Utilizes the brush tool to circle the unwanted toy.
Result:
- The toy is successfully removed with minimal impact on the surrounding textures.
- Some minor alterations, like the slight removal of pillow folds, are noted but deemed acceptable.

"It did a pretty good job of removing the toy without making it look too much like there was an edit there." [04:10]

Attempting to Remove a Person from a Building Photo

In a more challenging scenario, Micah attempts to remove a small person from the foreground of a building photograph. The initial attempt leads to unintended alterations in the stair steps, where the tool erroneously extends lines beyond their natural termination points.

First Attempt:
- Selects the person directly.
- Observes that the Cleanup tool improperly continues the stair steps, distorting the image.
Second Attempt:
- Brushes more precisely around the person.
- Despite closer selection, the result remains unsatisfactory with continued inaccuracies.
Feedback Mechanism:
- Utilizes the report feature to provide feedback on the tool's performance.

"Apple's cleanup tool is just not there yet. It is one of many features that are available as part of Apple Intelligence and some of them do a pretty good job. This one still needs some work." [10:25]

Cleaning Up a Spatial Photo

Micah explores the Cleanup tool on a spatial photo featuring himself, which includes additional objects like an iPhone and a keyboard on a table.

Object Removal:
- Attempts to remove the iPhone and keyboard.
- Notices partial success but identifies loss of texture and incomplete removal of elements.
Final Assessment:
- Decides to reset the Cleanup tools for the photo due to subpar results.

"It did an okay job of removing the iPhone, but you'll notice that some of the texture is lost from the table with the removal of that iPhone." [08:50]

Performance Analysis

Micah provides a balanced evaluation of the Cleanup tool, highlighting both its strengths and areas needing improvement.

Strengths:
- Effective in simple scenarios, such as removing large distractions from a photo.
- Maintains overall image texture and quality in less complex edits.
Weaknesses:
- Struggles with intricate details and smaller objects, leading to unnatural alterations.
- Limited by the on-device processing power, which restricts the complexity and accuracy of edits compared to server-side solutions.

"Apple's commitment to privacy and security run contrary to the tools that we see in use by other companies." [10:45]

Comparison with Other Tools

Micah contrasts Apple's Cleanup tool with Google's Magic Eraser, noting significant differences in performance due to their respective processing approaches.

Google Magic Eraser:
- Utilizes server-side technology with substantial processing power.
- Achieves more accurate and seamless object removal, especially in complex images.
Apple Cleanup:
- Relies on on-device models to ensure user privacy and data security.
- While promising, it currently lacks the sophistication of its competitors, resulting in less reliable outcomes.

"It gives you an idea of the difference between a company like Google, which has a fantastic magic eraser tool that relies in part on server-side technology and a lot more processing power to properly remove something from a photograph versus on-device training." [10:30]

Conclusion and Future Developments

Micah concludes that while Apple's Cleanup tool is a commendable addition to the Photos app, it is still in its nascent stages and requires further refinement. He remains optimistic about Apple's ongoing efforts to enhance the feature, anticipating improvements in future updates.

Availability:
- Currently available on macOS Sequoia, iOS, and iPadOS devices supporting Apple Intelligence.
User Feedback:
- Encourages users to provide feedback through email for potential feature enhancements.

"The good news is Apple is still working on these tools, working on improving them, and so hopefully we should see some improvements over time." [10:50]

Final Thoughts

The Cleanup tool exemplifies Apple's dedication to integrating artificial intelligence in user-friendly applications while upholding strong privacy and security standards. Although not yet matching the capabilities of some competing tools, its development signals a promising direction for AI-driven photo editing within the Apple ecosystem.

For listeners interested in experimenting with the Cleanup tool or providing feedback, Micah invites them to reach out via email for future discussions and feature evaluations.

This summary encapsulates the key discussions and insights shared by Micah Sargent in the episode, providing a comprehensive overview of Apple's new AI Cleanup tool, its functionalities, performance, and future potential.

Transcript

Micah Sargent (0:00)

Coming up on Hands on Mac, let's take a look at an Apple Intelligence feature called Cleanup. Stay tuned.

Tax Network USA (0:06)

Attention taxpayers. The IRS is intensifying collections in 2025. If you owe $10,000 or more or have unfiled tax returns, it's time to take action. Tax Network USA's experienced lawyers have saved clients over a billion dollars. They're ready to negotiate with the IRS on your behalf. Don't risk wage garnishment, property seizure, or losing your home cost. Call 1-800-958-1000 today. Get the help you need at 1-800-958-10000 or visit tnusa.com if you have a.

T-Mobile (0:37)

Locked AT&T phone, we're here with bolt cutters. T Mobile will help pay off your locked phone and give you a new 5G phone for free. All on America's largest 5G network. Visit t mobile.com CarrierFreedom via virtual prepaid.

MasterCard (0:51)

MasterCard in 15 days. Free phone up to 830 via 24 monthly bill credits plus tax and a 10 device connection charge. Qualifying port and trade in service on Go5G next and credit required for canceling entire account to continue bill credits or credit stop and balance on required finance agreement as deal bill credits and if you pay off devices early.

Bluehost (1:06)

I'm no tech genius, but I know if I want my business to crush it, I need a website. Now. Thankfully, Bluehost makes it easy. I customized, optimized and monetized everything exactly how I wanted with AI. Within minutes my site was up. I couldn't believe it. Plus, Bluehost keeps me protected 24 7. No more worrying about hackers. I just relax and focus on my customers. So whatever your hustle may be, secure your online presence in 2025 with Bluehost. Head to bluehost.com to start now.

Indeed (1:44)

Podcasts you love from people you trust.

Micah Sargent (1:48)

This is Twit. Welcome back to Hands On Mac. I am Micah Sargent and today we are taking a look at a feature that lets you remove distractions from your photos with Apple Intelligence. It's a feature called Cleanup and it sort of works. Let's take a look. All right, here we are in macOS Sequoia and I've launched the Photos app. And you'll notice that I have a photograph here of an adorable dog. This is Henry when he was a puppy and here he is beautifully, adorably, wonderfully resting on the bed. But in the background is kind of a little bit of a distraction. There's this toy that is there and I want the focus to be entirely on Henry. Now, of course, one thing we can do here is to simply crop the photo to a square, for example, such that the toy is removed. But instead I'm going to show you a feature for macOS Sequoia and for devices that support Apple Intelligence that will attempt to remove the toy. So we'll go into the Edit menu and you'll see that we have four options up at the top. Adjust, Filters, Crop, and Cleanup. You'll be familiar with Adjust Filters and Crop, but Cleanup is the new section and when you run cleanup for the first time, it is going to download to your system. This is sort of an on device model that has been trained for removing things that you don't want from a photograph. And it downloads this because it needs to do this this locally on device so that model gets downloaded and made available to you. And once it's been installed, you have a few options. You can click Brush or circle the thing that you want to remove and you have your pan and zoom features available to you. As always, you can change the size of the brush in the cleanup menu on the right. But in this case, excuse me, what I'm going to do is zoom in on the toy here and I am going to circle this toy and you'll notice a kind of over the top, in my opinion, animation showing you what you've selected and we can click to let go. And now it knows that I want to remove the toy. But as you'll notice in this case it didn't remove it. It instead thought that I wanted to take away the stuff that was around it. So in this case what we want to do is change the size of the brush a little bit and we're going to brush across the entire toy here. So we will do this to select the toy like so. And now we'll see the removal. So here is the before and here is the after. Before, after, before. After. In this case, it did a pretty good job of removing the toy without making it look too much like there was an edit there. The textures seem to carry along just fine. It did remove some of the fold of the pillowcase there, but that's okay. For the most part the toy has been removed and we can choose done.