← Software Engineering Daily
Software Engineering Daily
SED News: Apple’s AI Problem, The Real Business Model of AI, and Token Cost Reckoning
00:48:37
SED News is a monthly podcast from Software Engineering Daily where hosts Gregor Vand and Sean Falconer unpack the biggest stories shaping software engineering, Silicon Valley, and the broader tech industry. In this episode,
Loading summary
Transcript69 lines
- [00:12]
A
Hello and welcome to SED News. I'm Gregor Vand.
- [00:16]
B
And I'm Sean Faulconer.
- [00:17]
A
And I'm sure as many of you know, this is a different format of SE Daily released monthly where we touch on the main tech headlines from mainstream news. We go into a bigger topic in the middle but of debate in there and then go to Hacker News at the end and highlight some of our favorite things that have been posted. As usual, we like to just catch up with each other. Sean and I usually having quite busy schedules these days. You've been traveling, Sean? I think so. What's.
- [00:47]
B
I feel like every time we start these off it's like, oh, you've been traveling? Which seems to be the case over the last few months. Yes, we've kind of schedule these around my travel schedule. But yeah, I was in London this past week for our big Europe conference. It was a great time, saw a lot of people in this data streaming community. I was part of the keynote, gave a talk there. So it was a busy week but a fun week and we launched a lot of new products, my team. So it was quite a sprint leading up to the event. But it was nice to see everything come together.
- [01:16]
A
Yeah, nice. Well, I guess on the flip side, I've not been traveling since we last recorded SED News. I think I was in SF when we recorded. We met each other though. We actually had. You and I had lunch after that in person. First time we met each other, so that was always fun. I stopped back through Hong Kong on the way back, but that was nothing work related. That was just purely for my own enjoyment. Yeah, I'm back in Singapore and I'll be here for a few weeks, which is good. I think I do need less travel based on just how much is going on in the tech world right now. There's just my slack is just a deluge every morning. Every time I wake up with the time difference with us. It's an interesting time to be part of all of this as I keep telling people that are not in our industry especially. So given that we are in the industry and we do sit across quite a few things. So let's hit the main headlines. The first one is Apple. We haven't touched on Apple in a little while. Apple sort of at a crossroads. A lot of this comes from Financial Times did quite a good deep dive on this, but really the headline here is can Apple actually progress beyond, let's just say the iPhone for now? I think there are other facets obviously to their revenue generation which we can touch on briefly, but they've been this hardware business, they have gone into services and a lot of the last almost decade at this point is really just like just keep this thing running at optimum efficiency. And actually an interesting anecdote which feeds back to something that we did touch on at the time when their head of AI departed. But there wasn't a lot of information I guess around that. Exactly. But we have a bit more color on that now, which was the fact that the Siri demo basically had been fake when they had said, I think it was the last wwdc or like two WDCs ago, saying hey, this is the new Siri. And I think they actually put some supposed demo of it going on and turns out that was completely fake and the product hadn't been built yet. And they told their engineers, you've got nine weeks to get this out. And it didn't work really. And they actually got a 250 million class action lawsuit against false advertising, which I think that was the nail in the coffin for the head of AI heading out. It's just embarrassing, I mean this whole thing. But I think it's interesting this color, but it does start to paint a picture of. Do Apple really know what they're doing with AI? Like it's kind of interesting.
- [03:45]
B
So first of all, Salesforce has been doing vaporware demos for decade plus so and everyone kind of knows it and laughs about it, but they seem to be able to get away with it. Maybe it's because they're, you know, who they're selling to is not a consumer. So they don't end up with these class action lawsuits or anything with the false advertising. But yeah, I mean it's interesting, Apple hasn't to my knowledge ever had anything like really impactful in AI. Obviously they've had Siri for a long time, one of the first voice interfaces and so forth that was widely used. But in a lot of ways like voice didn't really take off the way that people maybe perceived it to do. You know, I worked on the Google Assistant at one point. It didn't really take off the way that we thought it was going to and it didn't end up being sort of this new modality that was widely used. Same with Alexa and stuff. Like a lot of these things became glorified ways of turning on and off lights and checking the weather and like really, really simple sort of low value tasks. But despite that, like Apple's like one of these companies that seems immune to what's happening in The AI world where so many companies that are public, their stocks are being punished right now even with like record breaking quarters because of the perception of how AI might disrupt them. But you look at like Apple stock and it's still, you know, it's like an all time high. It's still going really well despite them from my perception anyway, not really having kind of like an AI play. I think it would be tough though for them to become suddenly some sort of AI company. If you look at Google, Google always thought of itself as an AI company. Like Larry Page always considered Google to be an AI company. So it's not shocking. Even though they were slow moving compared to OpenAI on really monetizing and doing a lot around large language models. But they're well positioned to be very, very successful there. As we've talked about before, there just feels like a real cultural difference between who Apple is as a company and how they see themselves culturally versus how a company, like maybe Google or any of the AI native companies see themselves. So like what would it take for Apple to become that? Probably a lot of money spent on talent and maybe acquisitions. But even then, can you actually bridge that gap? And does it make sense? Because it's not just about like there's lots of problems worth solving in AI, but like how do you map that to what you're uniquely positioned to do well with is also a big question. So it's interesting because Apple's such a very successful company, has been for a long time now. People have a lot of respect for it still. It does really well in the market. It's one of the most valuable companies in the world. But it's kind of hard to see like what their next act is.
- [06:25]
A
Yeah, I think that's exactly it. And I mean this is it. Apple could absolutely decide to not lean into AI. I think they said something in their own legal filings they had actually warned that a truly capable AI assistant could make manually downloading apps obsolete. So obviously services, their app store are still quite a big money driver. If you look at their services breakdown for 2025, it's like almost 3x older Mac sales for example. So this is just still a huge part of their business. And I think there was also this interesting quote from this FT article which was back in 2010 when Jobs said he had acquired Siri, the company behind Siri. And people said, are you entering the search business? And he said, no, I'm not. I'm getting into the AI business. So I think it's one of these sliding Door moments is like what Apple might have become had Jobs persisted as CEO. Instead it's gone to this incredibly well run hardware business with supply chain excellence under Tim Cook, who's obviously retiring now or becoming chairman or something to that effect. And he's played this very well financially. Up at a billion sales per day, a trillion dollars apparently returned to shareholders 75% margins on services. But yeah, that is not innovation, that's a behemoth of money making. And their R and D spend I think has gone. It's like gone from 8% to 2% apparently during the iPhone boom. I mean, okay, 2% of a much bigger cash pile. Maybe it's exactly the same amount of money, but it's relative terms. You'd expect R and D to be keeping pace from a percentage perspective.
- [08:09]
B
Yeah, I mean there is probably something to. If you're focused on financial optimization, is that the enemy of innovation in some sense? Because I think innovation comes with a cost. Like you see that right now with the AI companies, we saw it with cloud, you see it in these early markets, you're spending a lot and not necessarily making a lot of money for eventually reaching some economies of scale in cost optimization in the future. And that might not play well into essentially the Tim Cook playbook for where he's putting his time and resources and so forth. So maybe that's something they have to change. I also think that on the AI front, Apple draws a lot of revenue today from app monetization. 30% I believe it is, that they're taking from app sales and so forth from a creator which is responsible for a lot of revenue. But if AI disrupts apps eventually, like if the entry point, sort of the face of the Internet becomes an AI app, that becomes like a super app, your gateway portal into like everything. And we've talked about this in terms of disruption to like E commerce, for example, going directly to an E commerce website versus just navigating to it through a chatgpt or something. If that becomes the interface for all applications, then that's going to hurt Apple massively in the long run.
- [09:32]
A
Yeah, exactly. So, I mean, yeah. Does AI saaspocalypse? Well, it's almost here. I guess it's apppocalypse. Literally. Where is a market for all these different apps? An app that I use is called Timeshifter, it's a jet lag app. But interestingly, my other half, she, for her own reasons doesn't want to pay for this app and she'd rather use Claude just to give Her a plan around jet lag. Now, I like some of the features that this timeshifter app gives me, but I'm sure my other half is not in the minority. That's like, actually, no, you know what? I don't need to pay for this app and I don't need to give, in that case, Google App Store a little cut of that one. I will happily just use my free cloud account and get a good enough plan instead. There must be tons of these examples that could really eat into Apple's app Store margins. So, yeah, it's definitely one to watch.
- [10:23]
B
I mean, it's been a while since what was Apple's last hit.
- [10:29]
A
It wasn't the Vision Pro, we know that.
- [10:32]
B
I mean, the iPhone, that was like 2007.
- [10:34]
A
Well, again, if you look at. I like the chart they put in the article, the FT article, the iPhone is still looks almost like 70% of revenue in 2025. So it is just unreal how they've managed to coast on. I say coast. Okay. The iPhone is still, I guess, one of the most advanced handsets out there, but relative to peers, it's much harder to distinguish now between what makes it so excellent. So, yeah, yeah, hits. I'm struggling. I'm struggling a bit on hits.
- [11:03]
B
I mean, there's, you know, there's minor hits, iPad and so forth, but nothing's. I mean, you're also comparing it to one of the mega hits of all time, the iPhone. So the bar was kind of set high.
- [11:13]
A
Yeah. So moving on, I guess one of the other big ones, Google, they had their I O conference just recently and yeah, I believe this was all the agentic pivot, if you like. I always think of you, Sean, as our resident Google expert. What were you thinking about this?
- [11:32]
B
I think like one of the things that I like them highlighting was the idea of these agents that are running sort of continuously in the background. Things like a personalized agent that can set up your work in the background 24 7, sort of monitoring different systems. And this is something that I've been talking about and writing about for over a year now, which is.
- [11:52]
A
This is the Gemini Spark, is it?
- [11:54]
B
Yeah, yeah. And this is something I've been talking about for quite some time of. We've kind of limited ourselves in terms of thinking of agents as these like purely chat systems. Like I asked the agent to go do something and it goes, works for a bit, comes back and gives me an answer and so forth. But there's lots of situations where you don't really want an agent to have to wait around for you to prompt it to go do the work. You kind of want it to just be able to like do the work because it knows something's going on. So I'm glad that they're sort of highlighting that and hopefully that's something that will help educate the world that an agent doesn't just have to be locked behind chat.
- [12:30]
A
Yeah, just jump to. There's another headline that we wanted to pull out here. Actually it was via PC Gamer, but it was that DuckDuckGo traffic has grown after Google dropping. Sorry, when I say dropping, I mean producing their AI mode. It's hard to drop a product or dropping a product. I don't know. But Google have defaulted to AI mode and you have to kind of go looking now for the classic if you like Google search, which was already under a lot of stress from people saying this is just too ad heavy, it doesn't do what it's supposed to do anymore. And DuckDuckGo, which is this alternative search engine, has actually seen a 28% increase in visits following the default to AI mode for Google, which that's pretty fascinating, I think.
- [13:19]
B
Yeah, I wonder how long that will last. Is this the gut reaction to change? It kind of reminds me of when Facebook first introduced the news feed and everybody was up in arms about this way back, annoyed by it, that they kind of rolled it out without telling anybody and suddenly there's newsfeed and people were upset about it and then they kind of got over it and now basically the news feed became social media. So I do wonder, is this kind of a blip reaction in temporary growth to DuckDuckGo or is this something more meaningful that will continue to gain traction? But I think that there is definitely some truth to this, like, obstacle course of what search has become. Like, you go and you do a search now on Google and you have your AI mode response, which if you click on, takes you into a chat experience. If you scroll past that, then you're looking at ads and if you scroll past that, you get the like, organic results. It's been a long time since I've been on the second page of a Google result. You know, a year ago, maybe it wasn't unheard of for me to go multiple levels deep on a Google search result, especially if I was looking for something obscure. Part of that is I'm doing a lot less Google searches these days because I go and I ask AI for more meaningful things and it can just give me the answer. But it's hard to see Where's Google going to go with this? You kind of either lean into it and you're just like, hey, we're a full AI chat experience now, like a ChatGPT. Or like, is this really what users want from search?
- [14:53]
A
Yeah, exactly. I think we did cover this in a chunkier section a few months ago in SED News. Sort of where we were looking at where will search go? We're getting a bit of signal then on this one, which is when Google tries to default to, I guess moving much closer to a. Call it like a ChatGPT experience, where the bulk of the result returned is sort of an AI response but with some context. Like for example, today I just searched for a specific company through the AI mode just to see what happened. And you get almost like a Wikipedia type answer, which just isn't what I'm looking for. And I'm like, no, I want to see if I'm looking in Google. Let's just say I am looking for, oh, what are the latest Reddit posts on it? I can just give me a list or I want to see any news results. Give me a list. I'd also just like to see what the website is and I'll go there myself. That's why I'm using search. So it is interesting how Google are just caught in this interesting spot right now where I don't think they really know actually which direction they should go in.
- [15:54]
B
Yeah, well, it's tough because they are in this place where they're obviously trying to stay ahead of ChatGPT and perplexity, these types of applications, but at the same time they're taking away from what the core value of what Google search was in the first place. And then it's this push pull where they need to try to protect future revenue, but they're doing it at the cost of the current user experience. It's kind of hard to know exactly what the end strategy is going to be. Yep.
- [16:28]
A
If you're running postgres in production, you've probably felt the moment analytical queries start fighting your transactional workload. Most teams end up adding a second database and all the pipeline complexity that comes with it. Tiger data creators of TimescaleDB takes a different approach. We extend Postgres with hybrid row and columnar storage. So one table handles both writes and analytical scans. Native compression cuts storage costs up to 95%. Continuous aggregates keep dashboards live without bash jobs and it scales to petabytes without you re architecting. Companies like Cloudflare, Octave Energy, Schneider, Axpo, and Flowco run production workloads on Tiger Data today. No stale data, no second system to operate, just postgres managed for you. Ready for the workload you're building toward?
- [17:09]
B
Try it free@tigerdata.com youm know Fidelity is
- [17:13]
C
a financial services leader, but did you know that inside Fidelity is a community of technologists working together to shape the future of finance and tech. Fidelity is always investing in tomorrow, from emerging tech to cutting edge tools that will transform what comes next. Their technologists are encouraged to keep learning so they can expand their skill sets, explore new ground and stay ahead of this rapidly evolving industry. And right now, Fidelity is hiring technologists to join their team. Fidelity technologists get the best of both worlds. Startup energy that's grounded in the stability of a financial institution. That means support, resources and amazing benefits. Bring your skills to a culture where you're empowered to dream big and build the tech that drives an organization and makes a real impact on people's lives. Find out more@tech.fidelitycareers.com that's tech.fidelitycareers.com Fidelity is an equal opportunity employer Most AI frameworks started with voice and bolted on video as an afterthought. Vision Agents by Stream was built video first from day one. It's an open source python framework that lets you build real time voice and video AI agents in minutes, not months. With 25 integrations for models like OpenAI, Gemini and Claude Sub, 500 millisecond latency on Stream's global edge network and support for YOLO, roboflow and custom CV models, you get a production ready stack without the infrastructure headache. Whether you're building coaching tools, multimodal assistance or real time security pipelines, Vision Agents handles the hard parts. Get started free at VisionAgents AI
- [18:58]
A
and then for our final headline this week, this is via TechCrunch. So remote, which is a company that I think quite a few listeners might be aware of and may even be recipients of their payroll through Remote, they've been growing pretty dramatically. So they are a 7 year old company based in Amsterdam. I didn't actually know that until I read this article. I thought they were American, but they've surpassed 300 million in ARR, which is pretty exceptional. I know compared to say an AI builder platform that says it's done 200 million ARR in a year or something, but I think this is a slightly more durable business I think people would agree with. But what is interesting is if we look at what's been reported is their per employee revenue basically. But it's just the fact that they claim that they've pushed through this 300 million ARR. Become cash flow positive, but they have not been hiring any more people. You could, of course, people could say this, but it sounds. This is CEO saying that their headcount has stayed flat to get to this place. So at least in theory, that proves that this is possible. I think we're also going to certainly if I scan ahead to my own hacker news highlights, it is going to be a bit of this coming in AI and people again. What does that mean? It should be more time spent at work, less time spent at work. At least I think you and I should. We're both seeing roughly the same thing in industry right now, which is more time spent at work. But okay, maybe headcount does stay flat, but that headcount is spending more time at work.
- [20:30]
B
Yeah, you keep it flat, but double the efficiency or double the output by having everyone work 80 hours.
- [20:37]
A
Yeah. And it is. Exactly. Yeah. I wonder if there's almost group dynamics at play where people see other colleagues achieving more and it just sort of keeps. When I say raising the bar, I'm not sure I want to say that in a positive sense. It just raises the expectation, I think is what I'm actually meaning, raises the expectation of output.
- [20:58]
B
There's probably some of that. If you're on a team and everybody's working extra hours and you're the one person who isn't, then it could potentially create some issues because your teammates might be like, well, why am I working 12 hours a day? And Gregor's getting away with less. So it can create some.
- [21:17]
A
If only that was true. But it's
- [21:20]
B
some social problems within the company. But I also think that there's also an addictive component to a lot of these AI tools. And I experienced it myself using quad code or whatever your agentic engineering tool of choice is. It is addictive. It's like a slot machine. You're putting your tokens in, it's churning away, you're getting some tokens out and you keep trying to get it to do more. And it feels like, hey, if I'm not using my compute cycles right now to cycle through tokens and generate code or generate something, then I'm just wasting all this compute sitting there idle on my computer. So you want it running all the time.
- [21:57]
A
Time.
- [21:57]
B
And it's like it's taxing. But it's a different type of taxing than being in full focus mode in like locked in coding. Like what used to sort of be a requirement to really be efficiently generating code. Now it's become somewhat of like an asynchronous process. It's kind of changed the mode of behavior and maybe that leads to you being able to do this for longer sustained periods than what we've been able to do before as well. There's a novelty factor. Like there's a lot of things I think going on that leads to the increase in people spending time. And it also is like a highly competitive market. People are getting laid off. There's pressure on companies to churn out more. It's got to come from somewhere. And then you have AI companies that seem to be moving incredibly fast and are very competitive against each other. Valuations are super high. There's lots of pressure all around. So I think that creates some of this as well.
- [22:49]
A
Yeah, absolutely. And investors I'm not entirely sure of remotes set up if they're fully VC backed or et cetera, but I'm sure they have to get to 300 million ARR. I assume they do have investment and they probably have investors looking quite closely at what does seem to be a bit of a new North Star. This per employee revenue per employee. I think I've mentioned this before. I still find this funny because of running a service business for 10 years and I would always look at revenue per employee. That's literally you're effectively selling time of a human to another business. And so you have to look at revenue per employee and then compare that to say one of our competitors who has 5x the number of employees. But if I did the maths on their reported revenue it's like, oh, we're making a lot more revenue per employee. I would rather be doing that than hiring 50 more people. But it's now become very trendy in tech to be reporting that and actually goner the low interest Years of boasting about just how many employees you have and how much you've grown their headcount and now it's almost the opposite. It's like boasting about keeping it flat or reducing.
- [23:53]
B
Yeah, I mean that becomes the new sort of North Star, I guess.
- [23:56]
A
Yeah. And it is hard to. I think for many companies this is all just the classic tech noise that comes out in the news. And I think every company just has to make their own assessment. They shouldn't be like sheep following just saying, oh well everyone's not hiring so we shouldn't either or whatever. Every company should make their own judgment on that. That's a fairly utopian view from me. I think just rounding out with what you were saying, Sean, about just the actual filling of time that an employee has. It is I guess like it's Parkinson's Law. The time available, whatever time is allotted will kind of always get filled. And I think we're just seeing that 100% where despite the efficiencies of the tooling and enabling us to generate the output in theory faster and potentially better, it just means we're being expected to output even more. And also when we talk about input, everything you output, at least in where I sit, everything that I output has to be taken in by somebody else on the team somewhere. And so we've got this debate going on right now of like, well, where's the line between you can't just keep churning out documents expecting a human to keep reading them? Or is it like AI versus AI on the input and the output or the output and the input?
- [25:11]
B
Yeah, you write a few short points and then use AI to explode it into like a document.
- [25:16]
A
Yeah, exactly.
- [25:16]
B
And then you give it to somebody and then that person puts it in AI to summarize it back into points so they can digest it.
- [25:22]
A
Yep, definitely have seen that. So I think that's a good place to leave that. Very interesting. We'll sort of follow along with how other. I like that this was Remote and not just yet another talking about meta or something. I think it's interesting to follow these companies that a lot of them are powering. I see a lot of the same names pop up power a lot of other startups these days and Remote definitely has played a big part in that. So yeah, very interesting. Moving to our main topic, we did touch a little bit on it in the last 10 minutes, but it really is just. Okay, well business models of especially coding agents and this is based on Simon Willison's blog post he published Put out something I think it was yesterday. And for those that aren't familiar Simon Willison, you'll see his posts pop up on Hacker news quite frequently. These very not always long but just very detailed dives into how especially at the moment, how certain models operate. He runs a bunch of experiments against them and so on, but he declared yesterday. I think Anthropic and OpenAI have found product market fit and this is based on the idea that subscriptions that suddenly the consumer pays has absolutely no correlation with a the cost of the business. But why is there product market fit? It's actually because enterprise is paying so much money for the use of these models. We have touched on this in the past a little bit sort of saying this all looks a bit strange. How can a $20 a month cloud subscription in any way be covering its costs? And why does my. If I look at our work subscription, those numbers look very different. And Simon Willison saying He's got the $100 a month max plan for anthropic and the $100 a month pro from OpenAI. He used a token usage tool over the past 30 days and he'd found he'd consumed 2000 worth of credits or tokens. So that's just to do the math there, that's 2000 ish worth of usage for 200. However, Enterprise are paying far, far over that. And that's the business model.
- [27:34]
B
Yeah, I mean essentially you get people hooked on this. It's almost like a freemium model or you could think of it. It's like open source or something like that where people for in their own projects and on their own free time they can use something that's less expensive. But then it's a way to get network effects across the industry because anybody can try it. And then once they try it, because they have product market fit, they're hooked. And people legitimately feel like they can't do their jobs now without these tools. I feel it myself, I depend on this so much now. So companies are willing to pay a lot for the enterprise licenses because their employees are pushing for it and then additionally had the market dynamics that we were talking about. There's just a lot of pressure on companies to try to generate more efficiencies and show that they're more AI forward and whatever it happens to be. So I think there's multiple things going on where you have both the bottoms up effect of people clamoring for this because they're using it themselves, they're paying for it with these cheaper licenses and they're pushing their companies for it. But then the companies are also sort of top down want these things. So it's very interesting dynamic which of course creates rocket fuel for all these companies. And I think in a lot of ways I don't think this was necessarily expected that this was going to be where the model companies found their true revenue driver for a long time is relative in this world. But not that long ago it was all about models, it was all about competing for the best model. And now that's really changed to competing for the best agentic harness. The true moat has lot to do with sort of the context management of the information environment around the model rather than just the base model. And we've even found in our own company that we can get similar performance from some of the lower cost models available from say anthropic versus the top tier models. So in a lot of ways the underlying models have kind of been at a performance level good enough for almost a year since about like November of 2025 I would say was when they kind of became good enough for tool differentiation. And you could. But the biggest change that's happened has been really like the management of that information environment around coding tasks which is completely transformed the developer experience over the last half a year or whatever. So that's a very different competition landscape than it being about bigger model wins, which is interesting. Like I don't think it's something that anybody necessarily saw coming. I also think that the other interesting bit about this is if you thought about like OpenAI and their strategy initially was maybe like, hey, we're going to own a sort of like become the face of the Internet. The way that Google became the front door of the Internet, that's a much different business model. Google is a volume based business at least from like search and ads and stuff like that. They get a ton of searches, they get a ton of users and they make a little bit of money off of every user and that leads to like a gigantic business. But if the way that I think there's like 900 million users of ChatGPT and they and only like a single digit percentage of them pay like that's an okay business but that's not like trillion dollar valuation business. And I think that where they're, you know, now seeing this is more of the B2B play. The anthropic I think really was figured that out even before OpenAI did. Now I think the comparison at least from a business model perspective is not anthropic OpenAI or the new Google, they're more like the new Salesforce or SaaS. You're selling seats and licenses, you're doing B2B enterprise sales. That's a completely different motion and completely different muscle, which is also interesting. It's kind of a deviation from maybe where people expected the market to go.
- [31:24]
A
Yeah, there was a lot of popped up in many news outlets the Uber CTO saying he'd maxed out the full year AI budget in just a few months in 2026 and that's definitely done the rounds. I especially in enterprise CFOs CTOs saying exactly, look, we've got to be super careful. This is Uber saying that they've risen to Their budget. I think the subtext there is, I don't think Uber would be saying that if they were worried that's what had happened. I think they're kind of saying it more like this is only positive, that actually we just made a miscalculation on how valuable these tools are and we happen to have spent what we predicted. But that's fine, we're just going to keep paying these companies. And I think that's where especially we might touch on it a bit. But the IPO is probably coming up with anthropic and SpaceX, which doesn't include Grok, for example. The way that Simon Millison put it, I think was very helpful. He says this isn't an AI failure story, it's just that a budget set in 2025, which is what happens at companies like an Uber of that size, the budget for 26 is set for far in advance in 25, basically failed to predict how indispensable these tools would become. And I think just also tied into this is the model selection. And to your point, Sean, actually the bit I'm curious about is that the models, as you say, the choice is less of a factor now. Like how what are the splitting hairs and what's the quality of output? Let's just say on Opus vs Sonnet, of course a lot of people say OPUS is far and away better and maybe there are some specific tasks that yes it is, but Sonic can still get you very, very, very far as it should be able to. But especially in enterprise, it's very interesting. I don't think a lot of people understand the default settings on their CLAUDE code or even when they go and just use Claude interface, there'll be some non technical users that are effectively given access to the enterprise account and they're just running everything through OPUS and not really paying attention. So maybe the average employee doing just some knowledge work through OPUS is like racking up, I don't know, $200 a month of usage. But we've got developers who basically want to say, look, I'm only going to use the best model because why would I use anything less? And then racking up like thousands and thousands and thousands per themselves. And I think companies, they're not saying don't do this, don't use this, but they're definitely starting to say, hang on, we got to keep looking at this because we want to make sure that at least use a cheaper model if you can. But I'm not fully clear on what the best way to address that is because you're kind of saying, if you think you can figure out that you'll still do as good work with a cheaper model, then go ahead, but if not, it's fine, use the expensive model.
- [34:11]
B
Now you're seeing more. And I actually talked to one of them today. It's like more startups that are focused on dynamic cost optimization where they're routing prompts to the correct or less expensive LLMs based on understanding sort of which models are good at certain tasks or looking at the history of interactions and so forth within the company. So I think there'll be more and more like there's going to be a lot of cost optimization stuff. But I feel like at some point, especially if the company with OpenAI and anthropic sort of stage to become public, at some point someone's going to pay for this. Basically. Like, you know, right now we're in it's like early innovation mode. Like we talked about. Companies are under pressure. They're not kind of worried about the token costs. They're more worried about the public perception that they're a dinosaur and they're not adopting these AI tools. But it's like suddenly all your employees are having a $25,000 token bill per month. What's that mean to your business? Like you're essentially getting like some sort of AI compute tax on every head that you hire within the company that you have to factor in. It's almost like in the US like factoring, okay, well we fully loaded employee, we got to pay for insurance, we got these other expenses that we have to. Now you're going to have a line item that's like what is the token usage of a senior engineer versus, I don't know, some other role, product marketing or something like that. You're going to have to be thinking about that as a company as well to factor this stuff in. So I think at some point there's going to be some impact, ultimate impact to either These companies are going to have to charge more under public, similar to what we saw with Uber or Lyft For a while, for a long time the ride shares were sort of VC subsidized and at some point there's no free lunches. Like someone's going to have to pay for this. And then that's going to also lead to, I think, a lot more cost optimization, which might start with sort of independent players and then eventually become somebody companies. We saw the same thing with warehouses as well. For the early days of when Snowflake was growing like crazy, people weren't really worried about their warehouse bill as much. But then when the market dynamics changed and things became a lot less about growth at all costs and more around cost optimization and margin optimization and stuff like that, then suddenly people started being very concerned about their gigantic Snowflake bill and other similar products. So then a bunch of companies and consulting services spun up around Cost Optim and then eventually Snowflake. In order to reduce the risk of churning, people built a lot of that tooling functionality actually directly into Snowflake so that people had more controls of how spend gets allocated and stuff. So I could certainly see things like that coming as well. But those are all signs of a more mature market and we're just in early early days. So people are just kind of spending blindly.
- [37:05]
A
And I guess tying it back to the headline about remote saying we kept headcount flat and it's because of AI. They obviously haven't. Well, I say they obviously haven't, but we didn't see any breakout from them of. But what are they spending on AI? If you're spending, I'm just picking numbers out here, but let's say as a business you have 100 employees and then you're spending 10k a month on these tools. Well, okay, what's that? That's at least three to four potential employees you might have had otherwise. A bit less, maybe two or three. It depends on the company, but that probably still makes a ton of sense. Don't employ two to people do put that kind of money into tooling for the other 100. Surely that makes sense. But as you say, Sean, we definitely haven't reached the real inflection point or the next stage in the journey, which is when do these costs start to come under scrutiny and when the cost optimization come in with cloud. Cloud is still the one that there's a ton of tools out there helping teams optimize their cloud builds on aws, et cetera. But if you look at the numbers, AWS still just keeps growing revenue and okay, people are much better at optimizing costs. But for example, at Supabase we have a team of two to three who their whole job is to optimize costs on aws. And I know that we're, I guess we're a little bit unusual in that respect, that that makes a lot of sense for us. But equally that's just symptomatic of the problem. Like you need to have three full time employees. If you use all of AWS to make sure you're not like spending money where you shouldn't need to spend it.
- [38:41]
B
Yeah, I think companies probably reach a similar spot with the model token costs or use and stuff like that. What do you think all this might mean for the open weight models? If the llamas of the world essentially, if the real value prop or moat is this agentic harness in the way you manage the context and information environment and not necessarily just the model, then what's the play for the open weight models? Do they have to go and find a similar sort of business to go after or can they survive just as a model alone?
- [39:16]
A
Yeah, well I guess to plug an episode. Spoke to Benny at Fireworks. Fireworks AI they run open weight models and eg, they run an open weight model for cursor. That's not all of obviously as people that I think that know use cursor it's not all their options. When you ask it to do something there's sort of like an auto mode which yes does run on these sort of tuned open weight models and then you can of course choose your specific non open weight models. I think it's going to be really interesting space because I think when more companies realize they could probably get the same outputs but then rather than paying a SaaS bill to just straight to anthropic, what if you set up your own again, call it like cloud instance on something like a Fireworks where you say hey, we know that this open weight model actually would be the best all rounder for the kind of work we do and we would end up spending 20x less over a year if you guys just run it for us. I think that's a very interesting space.
- [40:23]
B
Yeah, yeah. And the other thing too is these. The total tam of the space is gigantic. So there's going to probably be some model winner that or a collection of model winners that kind of own the open weight model inference market.
- [40:39]
A
Yeah. And at least from what I understand at present day, I think the thing that's interesting about open weight models is that just the tense, especially the bigger, more general ones, the majority are kind of coming out of the China basically. Miu Lama, yes. Is a standout from us. But how much more effort does Meta put into that? I'm not totally sure. It's definitely a big play. And all of these models coming from China which underpin a lot of some of the offerings from say Fireworks and we actually talk about that on the episode. I think there's just a lot to keep an eye on there and maybe actually that's a topic for a future SED news where we sort of look at how the open weight models are doing competing. So hopefully thanks to Simon Wilson for his very good article on that. I think it's very present, as we say looking at IPOs coming up. So we expect to see probably the S1 document, which is what gets produced before an IPO to attract investors. We expect to see that from Anthropic probably in the next few weeks, maybe months. Why is that interesting? Because that should have numbers break down. What is the actual revenue coming in from? Maybe they won't break it down precisely enterprise versus consumer, but we'll get to start to see at least what is the revenue coming in from usage and what are the costs involved with running such services. And that will be the SpaceX one. Don't think you'll have time to dig into that one today. But the problem with that one is it's a lot of financial chicanery, so it's a bit harder to dig into that one, at least from the AI perspective of what does it actually cost to run and how much revenue are they making? Just a sidebar is apparently the Starlink is actually the most profitable piece of SpaceX, or the only profitable piece of SpaceX is actually Starlink. So that's already an indicator of that one. But yeah, definitely don't have time to dig into that crazy S1 today. But maybe another time. It's that time of the show Hacker News highlights some of our favorites that have popped up. Oh, I see Doom. Doom's in there again. I love it.
- [42:46]
B
Yeah, yeah, I couldn't resist. You always got to highlight Doom on everything. But yeah, so this was running Doom on a travel router with touch, so it's essentially called a GL Inet Slate 7 Pro travel router. It's like a networking device with a 2.8 inch touchscreen. Apparently it runs some version of Linux and it has root SSH access. So to get Doom on there, I guess was relatively straightforward. But if you dig into the comments, there's some pretty funny ones. Like someone claims that they actually did this like 10 months previously, but I think one was there's kind of like an earnest thread about whether humanity should have picked a less violent game for this tradition of porting this one game across every type of device. Should it be Doom, or should it be something that's maybe a little less silent?
- [43:36]
A
Maybe someone needs to kick off like a Mario or something. Mario on something.
- [43:42]
B
Yeah, like the original Mario.
- [43:43]
A
Yeah, exactly. This didn't get very high on Hacker News, but it did the rounds and it's related, there's now something called Doom Bench. So it's like can your stack run Doom? So it's a performance analyzer of as it sounds, can your stack run Doom? So check that out as well if you're thinking of trying to port Doom onto something. But yeah, I love the travel router with the little screen. That's very fun. My first one is this was at the top of Hacker News before we were recording today. It's literally just called can we have the day off? And I thought this was a very timely. It's quite a short little article blog post by somebody called Mike and this was posted by a user called Mlsu. It's basically just sort of, well, if we're all getting 10x productivity from AI, can I just take Friday off now? Is that okay? I think probably a lot of us are feeling that at times. And this person also talks about they're paying $6,000 a month for childcare in California. So can I not just go into the office for four days and save some money on my childcare as well? Yeah, I think it's just I like these little zeitgeist articles that do actually make it up where I think it's usually a lot of what we're thinking in our own lives than someone just says it.
- [45:01]
B
Yeah, I think we talked about this throughout the episode, but I think there's probably a lot of people feeling like they're all running sprints right now. And how long can you sustain the sprint?
- [45:11]
A
Yeah, a marathon of sprints right now. Exactly.
- [45:14]
B
Yeah, exactly. It's a never ending sprint. The other one I had was YouTube is automatically labeling AI generated videos now. So before I think things were basically like an honor system. So like a creator could say that was AI generated and now they switching to automatic. So they're going to detect sort of photorealistic AI generated content auto apply labels. Even if the creators don't disclose it, it makes a lot of sense to me. Like I think we talked about this particularly with social media. If all the social media is generated by AI and then you're using AI to kind of like optimally sent show it to somebody and stuff like that. Like at some point like you know, who's this for? So at least I don't think they're changing the serving of it so they'll still serve you AI generated content, but they're at least letting you know that it's AI generated content. And I'm sure maybe there'll be more controls or something like that over time. But at some point you're going to end up with more content, not just on YouTube, but more content that exists on the Internet that's been generated by AI than generated by humans. And that may or may not be majorly problematic.
- [46:21]
A
Yeah, that's interesting. I would like to see that on Instagram. I don't spend a ton of time on Instagram these days, but it is still where I probably. Of any network where I go to just see what various people I do know are up to, as well as just random people I follow. But the AI thing is annoying because I see a bunch of stuff and I do have to kind of go to the comments to check if people say, obviously this is AI. I'm into birdwatching, for example. And there was a period when all these sort of AI birds started appearing and I'm like, that's not a real bird. Just no voting it down. So I would just like a little label and I can just filter out anything within certain topics. That is AI. That would be nice.
- [47:00]
B
Yeah. I know Dr. Christian Houbicki, who's been a guest on the podcast a couple times. Sometimes I don't know if he's still doing this, but for a while on Twitter, he would debunk fake robotics videos because people would always show some robot doing something ridiculous and stuff like that. And he would post his takes on it, like, of why this is like a fake video and things like that.
- [47:20]
A
Yeah, especially robotics. I guess you so get into the physics of it as well. Like there's no robot that could possibly do this or something to that.
- [47:27]
B
Yeah, exactly.
- [47:28]
A
So cool. My second one is. I feel it's like quite vanilla, but I quite like it. It's SimCity 3000 in 4K. So someone that has put together a nice port of what it sounds like, SimCity 3000, and you go to GOG, which is a sort of game, has game exes that you can grab and you grab that and you do a bunch of tweaks that this person has put together. And then it runs in 4K, which is super nice. So you run it on your nice widescreens, et cetera. Interestingly, I always liked the genre, but I ended up playing a game called Cities Skylines. I've never quite understood why it's not Cities Skylines, but I believe it's like a Finnish developer. So it ended up being called Skylines, but it was an excellent sort of. The first one wasn't 4K. I believe there's now a second Skylines 2, which is 4K. But then it is nice if we can go back and play these classics that we're so used to, but just with uprated graphics. As a sidebar, I did end up downloading Metal Gear solid for the PS4, which like the original and they actually have two modes, like you can run it in original graphics mode and. And improved graphics mode. And I've got to say I actually almost prefer the original. Just it's like this kind of fun, blocky experience that reminds you of how PlayStation used to be. So each to their own on whether they want to uprate, see their classics uprated, or just play it as they were.
- [48:53]
B
Yeah.
- [48:54]
A
So then, yeah, just any thoughts for what we might see across the next month when we check in next?
- [49:01]
B
The only thing that jumps to mind is, and I think maybe next month is probably too fast, but I do think that we're going to see a lot more around, like token cost optimization. I think maybe you look ahead six months. I think that's where we'll start to see that, where companies will become either more concerned about it or we're going to see a lot more sort of startups coming on the scene that will help you sort of manage your token cost and token bill that you're now spending on Claude. Because at some point can you really justify, I don't know, a $25,000 or whatever it is per month bill of everybody in your company? Maybe, maybe not. I guess it depends on what the efficiency gain and the ROI is on that. But that's a pretty high bar for getting ROI from everybody at that level.
- [49:47]
A
Yeah, I guess sort of related, but maybe I'm taking a slightly alternate take and I'm probably ahead of my time at this one, but S1 and S1 from anthropic, I think we might see that. And if we do see it, I'm going to stick my neck out and say people will be very surprised, positively at how much money they're making and say that this is the next Microsoft or even the next Nvidia to some degree. Yes, the R and D costs have been massive, but this is clearly going to be where all enterprise money ends up for the next 10 years.
- [50:20]
B
Yeah.
- [50:21]
A
Yeah. Let's see what happens. As usual, thank you everybody for tuning in. We've had a few notes from various people that this is SED news is what they look out for on a monthly basis. So we really appreciate that. Both Sean and I have had separate anecdotes from people we've met. So anything you would like to see covered. Always do. Just drop us a message on various sources, socials, and you can find se daily on LinkedIn, X and all the usual places. So do give us feedback. We love it.
- [50:48]
B
Yeah, absolutely. Feedback's welcome.
- [50:50]
A
Yeah, cool. All right, well, until next time. We'll catch everybody on next month's SED news.
- [50:54]
B
Thanks, everyone.