Loading summary
Transcript59 lines
- [00:00]
A
Foreign welcome to App Stories. I'm John Voorhees and with me is Federico Fedici. Hey Federico.
- [00:19]
B
Hello, John. How are you?
- [00:20]
A
I'm doing really well. Happy Friday. We are recording this on a Friday. This is, you know, the behind the scenes bit of the show.
- [00:25]
B
I think, you know that. I don't know if I ever told you this, you guys. In America you have Friday the 13th as your unlucky day, right? In Italy we do, we have Friday the 17th which is today because you.
- [00:40]
A
Want to be a little different there in Italy. Is that the, the thing you want to do?
- [00:43]
B
We say, we say in Italy, we say Vinardi Dicha Sete. That would be Friday the 17th. That's our national unlucky day. I don't really believe in these things, but a lot of people do. You see like, you actually see like a lot of people like not doing particular things like not leaving for, for a trip, for example, not taking the bus, not riding a bicycle, just kind of staying home. Yeah, I don't want to do anything today. I don't really believe in that and obviously, you know, I say that but like, you know, knock on wood, there's still a few hours left.
- [01:19]
A
Yes, yes. It's still early in the day. Well, be careful out there, Federico. We hope nothing happens.
- [01:24]
B
It's like people think that black cats are a sign of misfortune, but I think black cats are beautiful. So all cats are beautiful and dogs obviously. That being said, we talked about Android and some specific Apple related things in the pre show for app stories plus members. We have a completely different topic in the main show which is we're early in the year so, so maybe, you know, it's time to start thinking about the things we're going to see in June. And I have a bit of an Apple intelligence wish list that I wanted to share.
- [02:00]
A
Very good, very good. I think that there are a lot of things to wish for with Apple intelligence because we've gotten so far is, you know, just the surface stuff. I think the big ticket things are yet to come and I think we should add to that big ticket list.
- [02:14]
B
I think we've each put together a list of things we would like to see and we're going to connect it style round robin. This I am going to start right off the bat with the first one which is I hope that this year in iOS 19. Yes, 19 this year. I hope that in iOS 19 we are going to see a more conversational Siri in the sense of credible ChatGPT alternative in the form of a Siri app that lets you use a large language model to have ongoing, persistent conversations with Siri, just like you can have conversations in natural language with ChatGPT. I hope Apple will showcase, and I don't think they're gonna release it this year. I think it's gonna be a 2026 thing, but I hope we're gonna see this sort of experience. I think this is complicated because on one hand, I think it's going to be a tall order for Apple to even match the functionality of ChatGPT. From two years ago, we know that, pretty much established that Apple intelligence is roughly two years behind the competition. You look at ChatGPT, you look at Gemini, you look at Claude, and it's very clear that the Apple intelligence that we have today is basically where the rest of the industry was in 2023, arguably even 2022.
- [03:39]
A
Right. And right now, none of it at all is baked into Siri. Because just yesterday I asked Siri what. What's the weather going to be in two hours? And instead of telling me what the weather was going to be in two hours, it told me what the weather was going to be at 2pm, which was three hours before the time I was asking.
- [03:56]
B
So why not, why not, why not, why not? And so I'm going to lump this together with the next item in my list, which obviously this sort of experience that I imagine as being rumored by Mark Gurman at Bloomberg. The idea of a Siri LLM. The idea of a Siri large language model that allows you to chat and to have longer ongoing conversations, persistent conversations in natural language with Siri. I think even though I said it's going to be a tall order for Apple to do it, the thing is, I believe Apple has to do it. You look at ChatGPT, you look at the App Store, ChatGPT is consistently in the top five most downloaded free apps on the App Store.
- [04:41]
A
Right.
- [04:41]
B
Like, and I know I'm going to wager that it's the case everywhere I look at the US App Store, I think it's the same everywhere. ChatGPT, the ChatGPT app has become this wildly popular global phenomenon that I see on everybody's iPhones these days.
- [04:58]
A
Yeah. And the fact of the matter is that the, the fact that Apple Intelligence incorporates ChatGPT isn't a good solution right now because it doesn't have the same kind of persistence and memory and continuity that you're talking about that the app has.
- [05:12]
B
And it's not. And it's not frictionless. Right.
- [05:15]
A
Right.
- [05:15]
B
It's built for these sort of like ephemeral requests where you ask one thing and Siri realizes oh no, I don't know how to answer this. Do you want to go to ChatGPT? And you go to ChatGPT, but it's not the full experience and at that point I think you're better served by actually opening the ChatGPT app on your phone and having a proper conversation there with more features and faster speeds. So this has a lot of potential ramifications. The first one obviously if Apple is doing a large language model, what's it going to look like? Look at the rest of the AI industry, look at OpenAI, look at Google, look at the open source scene, look at Llama, look at Quantum coming from China. All these companies are releasing either closed source or open source models and they release a collection of models. Usually models are, I'm going to simplify here, models contain a certain numbers of parameters and they support a variety of context windows. The context window being the amount of text that can be stored in the same conversation with a large language model.
- [06:21]
A
Right.
- [06:22]
B
OpenAI, Google, anthropic, they have different flavors of their models of ChatGPT, Gemini and Claude. There's usually a bigger model that contains a lot of parameters, has been trained on a lot of parameters, a lot of text and supports a very large context window. Famously Google with Gemini is the only company to offer I think at the moment a 1 million token context window which, which is very, very much higher compared to, to, to ChatGPT and Claude. Now I'm thinking is Apple, if Apple is going to do so, let's say Apple is going to come out and say we have a Siri LLM first of all, are they going to follow the rest of the industry in having multiple versions of Siri LLM. Is there going to be? Because I could see a scenario in which Apple following their own well established and copied in. If you look at Dell naming pattern, I could see a scenario in which Apple does Serial M Mini, Siri lm, Serial M Pro and Siri Pro Max maybe.
- [07:29]
A
I think what they're going to do is they will have multiple models, they're just not going to tell you about it because I think because it'll be simpler, it's a marketing thing, the Mini one will be on your phone, it'll be local and it'll be used whenever possible to get you answers really quickly. And then the bigger ones, it'll go out to the web and it'll go out to that, you know, Cloud compute that Apple's implementing.
- [07:50]
B
Yes, I think you're right. I think you're right. I think we could see a scenario in which a, I don't know, a 1 point, 1.5 gigabyte model, the mini model is stored offline on your phone, right. And that could be serial mini. It works offline and it handles basic requests, but for other bigger models, it goes off to private cloud compute and so it requires an Internet connection. I think we're years away from this sort of model. I think it's going to be a 2026 thing. That's not even to mention a reasoning model, which I think if Apple is ever going to have a reasoning model, that's going to be like, at the very least like a 2027 thing. Right, but additional questions, are they going to support multiple modalities? By modality, I mean the type of input, the type of interaction that you want to have with a serial alum, are you going to be able to talk to it in real time like you can with Gemini or ChatGPT voice? Are they going to support text? It seems pretty much a given. Are they going to support image attachments? Are they going to support local document attachments? And the other big question that I have, the obvious advantage that Apple has compared to the competition is the fact that they own the platform, they own the ecosystem, they own the devices that we use. And that means that Siri and shortcuts, all these features, Apple Intelligence as a whole has access to your apps, to data from your apps. The thing that ChatGPT cannot do is look into my reminders, looking to my Apple notes, looking to my Obsidian. Theoretically that's something that Apple Intelligence could do. That's something that a Siri large language model could do. And when I think about this, and I think about the potential scenario in which Apple is coming out with the Siri LLM, I get very excited about the idea of chatting with Siri about the contents of my Obsidian, talking to Siri about the contents of Apple Mail.
- [09:44]
A
Yeah, look, I think that where we're heading with this is that Apple Siri for Apple is going to become another interface for the OS itself, that we are going to have apps, there will be apps that will incorporate heavily artificial intelligence, but the system itself is going to incorporate it too. So it's not just going to be go to your Siri AI app, it'll be go to Maps and ask it a question. That is more the kind of question you'd ask to an LLM and find out information about locations around you, or you do the same in notes or reminders or whatever app it happens to be. And this kind of feeds into my first thing, which is I want to see a dedicated research and writing tool integration. Now I could see this being a separate app, a little bit like NotebookLM, that Google has something along those lines where you can, you know, multimodally add documents and audio and other things to a collection of things in a project and then use it to mine it for information, for research. But I could also see it just becoming like a sidebar in something like in something like pages, for instance, or freeform, something like that, where you've got other information that you're creating, whether it's text or a mind map or maybe a spreadsheet and numbers, and you want to incorporate other information where a large language model would be useful. So I think as a research and a research tool that supports the existing productivity apps, that could be really powerful.
- [11:29]
B
Yeah, as you were mentioning this, I thought the idea of potentially sort of like being able to have a super brain where you could reference almost like deep link specific points of multiple conversations that you've had with the Siri LLM. Like, let's say you're working on a project, right?
- [11:52]
A
That's a good point.
- [11:53]
B
You're working on a project, you're working on your next macOS review. And you know, you want to reference, like, maybe at some point you had a conversation with Siri about like, what are the different names that Apple use for macOS versions? And then another point you ask, hey, when did Apple add the new design? That was macOS Yosemite. But like, you have these multiple conversations and being able to reference those points of multiple conversations in the same project, like, that's something that. That's something like that, that kind of like instant and total recall. It's something that the human brain doesn't really support unless, like, I mean, do you remember, John, exactly how every single conversation in your life has gone?
- [12:33]
A
No, no, obviously, right.
- [12:35]
B
You remember the gist of it. You remember maybe some. Some of the ideas, but obviously we're not. The human brain is not. Doesn't scale like that. So the ability to have that sort of research tool based on your data, I think it's like the ultimate assistive AI tool.
- [12:52]
A
Yeah, I think it is. And you know, as a result, you'd also want to have it built into Safari as well. I mean, just the other day I was writing a story for the club all about the camera and the rumor that the 48, there's going to be a 48 megapixel camera sensor in the iPhone SE. And I was trying to compare what different sensors were incorporated into what Apple devices on what dates. And it's all over the map. Whether you're talking about the iPad or the Mac's FaceTime camera or the iPhone's rear or front facing cameras, there's a ton of camera sensors in all of those devices. And plotting when the 48 megapixel part came into view and where things are heading on each of those platforms is something that's just like not in my brain because there's too many different factors and different models to keep keep, you know, remembered in one place. So I was using ChatGPT for that and it was perfect for that because then I could get the links and I would just go straight to the Apple tech specs to confirm that it wasn't making anything up. But it was just a really shorthand way and very quick way to research as opposed to like scrolling through page after page of tech specs, finding the tech specs pages in the first place and then going through them and comparing them side by side, I could very quickly get a summary, click through to those links, verify everything was right, and move on with my writing. It was much faster than traditional web search that way.
- [14:19]
B
Yeah, yeah, it's like, I think if Apple just wants to copy ChatGPT and have like something that you can talk to and ask questions about like web sources or general, general, like general knowledge, I think that's fine. But I think the real potential is something, something along the lines of a mix between ChatGPT and what Microsoft is doing with their Rewind feature, which it's sort of available now. It was like off to this pretty much terrible launch where it was like a privacy nightmare and then Microsoft sort of went back to the drawing board for that. But the idea of, you know, being able to privately and securely access data on your computers and interact with apps, because that's the other thing I really want to see what Apple does with these app intents, with this ability to ask Siri to perform actions in your third party apps without having to put together a shortcut first. I'm going to get to shortcut in a minute. But I get very excited about this idea of Apple actually catching up with the competition and actually eclipsing the competition in a way that only they and maybe Google can do. Because obviously Google owns Android. And I could see Google, I could see Google sort of beating Apple to the punch at Google IO with similar features in many like having a framework to actually plug Gemini into all of.
- [15:44]
A
Your Android apps and Chrome OS too for that matter.
- [15:47]
B
And Chrome os. Yeah, yeah, I could see, I could see Google and, and I mean Google has pretty much said we're going to do agents in Google Chrome. But yeah, I think it's an exciting space to watch this year and I hope we go beyond, you know, we saw image playgrounds and photos cleanup, image 1 like this, pretty rudimentary tools with the first wave of Apple intelligence features. I hope we go beyond that.
- [16:11]
A
Yeah, I hope it goes beyond voice too because I mean I think, I think voice is an important aspect of this but. And maybe this gets a little bit to what you're going to talk about with shortcuts, but I think that there is a place for one off actions asking Siri to do things on your behalf with your apps and your data on your computer or your iPhone or wherever. But I also think that Siri, and Siri as an AI could be a big help in actually creating reusable automations, you know, that are multi step things that you don't want to have to ask it to do all over again. It actually, I mean because that's something I've been doing with ChatGPT is it? It's a really good tool for helping work your way through a shortcut idea, you know, giving you, telling you what the actions are. It does hallucinate a fair amount with, with shortcuts I think because it's a kind of a visual app. But I think that's something that Apple could do really well with shortcuts is is use it as a tool for helping people, walking people through the steps of finding the actions whether they're on, you know, built, whether they're the built in ones or whether they're third party ones that are available either from apps that those that someone's already installed or maybe even ones that are that it knows about because it has the entire database of the App Store out there and what's available in terms of shortcut actions to give people suggestions about apps they might want to use to support their automations.
- [17:34]
B
Yeah, I think I mentioned this before. I am convinced this is just my personal theory that the spin that Apple is going to put on the AI agents that everybody's doing now, Apple spin is going to be shortcuts. I'm convinced that at some point we will be able to create an agent that is basically a fancy new kind of personal automation. Essentially like the whole idea of Agents is to use a natural language. Use a natural language and a large language model to have it perform something in the background for you. And we're still, we're seeing these kinds of agents in IDEs and in web browsers where you tell the agent to do one thing like book me a table at this restaurant. And it's literally a large language model using image recognition and UI scripting to. To click around the web browser and do things for you and then ask you for confirmation. I think Apple has the potential here to actually do native actions because they own the app and the framework that is shortcuts and app intents to perform tasks, to do stuff in your favorite apps. But the interface is clunky and personal automations are even clunkier and they're not well supported across platforms that are still iOS and iPados only, and they do not sync. I could see a scenario in which Apple says you can create agents with Apple Intelligence. They're based on shortcuts. I could see Apple renaming shortcuts to agents even in their quest to catch up and to put up a good face and be like, we do AI. I could see Apple renaming shortcuts to agents.
- [19:25]
A
But Apple agents, how about Apple agents? Right there, we got Apple Intelligence. We have Apple agents.
- [19:30]
B
Right, we got Apple agents. I could see that sort of feature becoming something based on shortcuts and based on app intents. Where you say, I want an agent that obviously this is going to require a lot more app intents and a lot of more triggers and features that do not exist in shortcuts now.
- [19:50]
A
Right.
- [19:50]
B
For example, an agent that says, whenever I get an email from John, mark it as important, or whenever I save a PDF to this folder, rename it with the current day. Like something along those lines. Like an agent for. You can apply these to photos, to Safari, to mail, to Calendar, to whatever you want. Anything that integrates with shortcuts, I think is a prime candidate for becoming a little agent that lives on your devices that is powered by Apple Intelligence and powered by a Siri large language model.
- [20:20]
A
Yeah, you know, I think that's a great vision and I think that I'd love to see that too. The problem I have right now is that there are so many aspects of the system that have not been turned into actions already, especially when you look at the Mac. I mean, these, these agents are going to be powerful everywhere, but I feel like shortcuts really is still stuck in the mud when it comes to macOS.
- [20:44]
B
Yeah, yeah. The sort of. The second part of My agentic wish list is I hope that we see more and more from Apple intelligence in terms of multitasking and productivity. It's nice to have conversations, it's nice to have photo related AI features. I think there's a lot of potential. And again, Apple is well suited for this sort of thing because they make computers. OpenAI doesn't make computers, Google does. And so I think Apple, Google is actually Apple's main competitor in this space. But for example, allow me to spend less time managing my files, renaming files or organizing them in folders, allow me to use a larger language model to, I don't know, assemble a workspace for me. And yes, I know nerds will always want the ability to manually arrange their windows, move them around. You know, John Syracuse is going to lose his mind. But like, I know and I'm probably still going to want to like carefully arrange my Windows in Stage Manager, but the idea of like say assistant, when I sit down to work, open the windows to let me browse the web and take a note and also show me the calendar on the side like instead of having to recreate that workspace every single time, essentially what I think Apple is going to do, I think this whole Apple Intelligence thing and this whole sort of AI catch up session that Apple is going to have for the next few years I think is a new kind of fuel that will allow Apple to think of new features across the system. Past few years, if you think about it, what have we really seen in iOS and iPados that's truly new? Beyond customization, like there's only so much you can customize now. We have wallpapers, we have lock screens. Most of the actual OS innovation has happened in customization for the past few years. I think this whole thing with AI is an opportunity for Apple to fundamentally rethink how we operate with our computers. And so I hope they go beyond Genmoji and Cutesy horrible for me images and actually embrace productivity, how people are actually working with their computers.
- [23:15]
A
Yeah, I agree with you on that. I mean, here's the thing. I mean, I know you say that maybe the nerds will want to arrange things just so, but I think that AI can help there too. Because what you should be able to do is arrange that screen exactly the way you want it and then tell your Siri AI this is the way I want to work in Safari and Obsidian every time, create, you know, save that for me and then be able to launch it and repeat it and have it repeatable every single time. That's Essentially what people do with things like Keyboard Maestro and you know, better touch tool on the Mac, for instance. So I think that that makes a lot of sense. I also feel like there's a lot of opportunity for working with the apps and the data that are already on your devices. But I'd also like to see this AI be able to work with data that's out on the Internet. So I want to be able to say I want to go to a website and see a giant table of information, say Siri, create a spreadsheet for me using the data in this table on this website, that kind of thing. Whether it's information that's displayed on a webpage, whether it's a file that's linked on a web page, whatever it happens to be, be able to do that and create that stuff for me or tell it, go to this website where there's API documentation and explain to me how I can integrate this with shortcuts. Because I think that those kind of things are going to be really powerful. Plus there's a developer angle here too, which is I know that a lot of developers I've talked to aren't super keen on SwiftAssist, which is the, the LLM auto completion of swift code that has been added to Xcode. And so I think there's a lot of work that can be done there because we're seeing apps like Cursor, which is incredibly powerful and also very popular. So I think just as Apple is a little bit behind on the large, large language models in general, it's also behind in kind of the code aspect of this. And that is important to keep developers happy. So I'd like to see that moving forward quickly too.
- [25:25]
B
Yeah, yeah. Developer tools are changing. It's just, it's just a fact. You just look at what people are doing with Cursor. It's, it's, it's wild. Speaking of developers, here's a question for you. Is there going to be a Siri LLM API or. Because I would think like right now there really isn't an Apple Intelligence API. I would think that at the very least there should be a native API for apps to handle processing of data. And you know, just like look at how many iOS apps you see that have a ChatGPT integration that maybe lets you rephrase text or in, in an automatic fashion.
- [26:01]
A
Right.
- [26:01]
B
I think there should be a serial API. And here's a follow up question. Is there going to be a web API? Are you going to be able to make web requests to a Serial or not.
- [26:11]
A
Yeah, that would be interesting. I think there are a couple aspects of this that I could see. One is, is structuring your data for your apps in a way that is optimized for a Siri LLM. Just giving it a little bit of a head start as to what kind of data it is and how various ways it can be used with different actions that are available to the LLM, whether it's through shortcuts or app intents or whatever. That's part of it.
- [26:36]
B
That's basically what they've done with the app intents, schemas and domains they have.
- [26:42]
A
But I think that that needs to expand because right now we're at maybe a dozen or so.
- [26:47]
B
12, I think 12 categories they have.
- [26:50]
A
Yeah, yeah. So that, that needs to expand. But you're right, it's the kind of thing where it's not just about structuring the data in the apps. It's a question of whether third parties can use Siri itself as a way to optimize things or use it inside their app. Because, you know, app intents are more about sharing between things. This would be more an API for acting on the data that's already in your app. So, I mean, we'll see, we'll see.
- [27:19]
B
Yeah. My final thing is just, it's more simple. I hope we see some kind of pixel meter or photo meter functional, inspired functionality find its way to the Photos app with some new AI features like the super resolution features that they have, the machine learning adjustments that they have in pixelmator and photomator. I think those would make for pretty nice additions to Photos. I really want to see what Apple does with those apps that they now own this year.
- [27:55]
A
Yeah, yeah, I think Photos has some of that built into it already. But you're right. I think pixelmator goes. Pixelmator Pro in particular goes a step further when it comes to things like resolution and some of the other tools that are baked into there. So it'll be interesting to see how that gets incorporated. And you know, there are things that Apple could do, maybe, I don't know, expand how tagging works, different kinds of automation to take kind of the tedium out of managing a photo library, basically.
- [28:23]
B
Yeah, so that's my wish list and you know, watch as none of these happens and Apple instead does image playgrounds too. So.
- [28:33]
A
Yeah, we'll see, we'll see. I mean, there's a lot to be done and I think you're right. I think it's to set expectations. I think this is not just a 2025 thing. I think that you and I will be talking about this for the next two, three years and beyond, even from there. So it's not slowing down. And hopefully Apple can kind of both catch up but also innovate itself and do some things that some of these other companies haven't done yet. I think it's time especially and I think Apple is particularly well poised to break free from the chatbot. I think a chatbot has a place in these systems, but I don't think the chatbot is necessarily the solution for all UI for an LLM. And with, you know, access obviously to the oss themselves, Apple has an opportunity to do that better than I think any of these other companies.
- [29:23]
B
Here's an idea. A foldable iPhone that opens itself thanks to AI.
- [29:31]
A
Okay. Federico. It launches Vivaldi, the browser.
- [29:36]
B
It opens and closes with AI. It's a robot iPhone.
- [29:41]
A
Then you tell it to install Android on itself and become a pixel phone. I don't know. All right, we better end it there. Federico I think you can find both of us over@macstories.net, we are at Club MacStories 2. That is our subscription service. Club members get all kinds of stuff. They get weekly, monthly newsletters, depending on your tier. You can join our club Discord. We do special things with our podcast Mac Stories Unwind plus, as well as the plus version of this show which comes out early and is extended and ad free. So there's lots of great perks there. Check out all the details at Plus Club. And of course you can find us on our other shows, Mac Stories Unwind and NPC Next Portable Console. Talk to you next week. Federico.
- [30:32]
B
Ciao.
- [30:33]
A
John.