← Software Engineering Daily
Software Engineering Daily
SED News: OpenCode, AI Code vs. Shipped Code, and the LiteLLM Breach
00:56:42
SED News is a monthly podcast from Software Engineering Daily where hosts Gregor Vand and Sean Falconer unpack the biggest stories shaping software engineering, Silicon Valley, and the broader tech industry. In this episode,
Loading summary
Transcript62 lines
- [00:12]
A
Hello and welcome to SED News. We are your hosts. I'm Gregor Vand.
- [00:18]
B
And I'm Sean Falconer.
- [00:20]
A
And as I think a lot of you do know already, this is a slightly different format of Software Engineering Daily where we pick off some of the big tech headlines that you might have seen in sort of more mainstream news. We're going to dive into a bigger topic in the middle and then we just look at some of our favorites from Hacker News towards the end, which usually ends up in sort of weird and wonderful rabbit hole type projects that developers have been working on. So as usual we just like to kind of catch up on what we've been doing. What has been in your sort of sphere, Sean, since we last did a SED News last month?
- [00:56]
B
There's been a lot in the last month. So I moved into a new house which has been great. But while I also moved in a couple weeks ago, I actually have traveled each week of this new adventure in this new house. So I was in D.C. and then I was in Seattle for our data streaming world tour last week and then my wife was away this week. So we've been here almost three weeks and there's never been a completely full week where everyone as a family has been in the house yet. So I'm looking forward for that happening and then confluent. Also, the company I work for was officially acquired by IBM about a week and a half ago. So that's exciting news.
- [01:32]
A
That's all completed then.
- [01:33]
B
Yeah, yeah, everything's been completed. So lots and lots of conversations now going on with those folks over at the IBM side and figuring out how we're going to work together, which is exciting. Lots of stuff happening and then you're looking ahead to April. My kids are going to be on spring break, which means that everybody's going to be on spring break for vacation. So we're, we have some travel coming up and then I had some work travel to Cloud next and then I also going to India. So lots and lots of stuff happening over the next little while. But what's going on in your world? You're in a different location than usual.
- [02:04]
A
Yeah, well, I'm in Scotland, so it's where I'm from originally. I try and come over here for a month or two of the year. When I do, I'm up in the far north in the Highlands of Scotland. So it's always kind of fun. That's what technology makes possible. You can still do a podcast from all, all the way up here where there's I Actually looked it up. The town I come to is a population of 88. 0 people. So I thought I was from a small place. Yeah. So I didn't actually grow up here, but it's sort of where I've decided is where I come back to now when I come back to Scotland. So, yeah, it's always quite a change from Singapore, but that's, I think by design. But yeah, no, I mean work wise has been kind of exciting actually. Obviously small plug here for Supabase, of course. But yeah, we've just launched. So Stripe have been doing this interesting project called, well, called Stripe Projects, which is a CLI tool effectively. So you install it through their CLI and basically it's a bunch of partners. It's not just Supabase, you'll find all the usual names on there, but it's been really great. I've been kind of overseeing that one and it got out yesterday. So we're recording this on a Friday and this became kind of public in preview yesterday. It's just awesome when you see something actually hit the public sphere that you've been working with engineering on for the last. Like we've really been crunching on this the last couple months and nice to see that go out. So, yeah, check it out. And yeah, I'll be in SF later this month as well. And that's for sessions. That's Stripe sessions. So for obvious reasons, as I've now just disclosed, we're working a lot with Stripe and stuff. So, yeah, if anyone is at Stripe Sessions, do look me up and say hello. Always great to meet new faces as well.
- [03:37]
B
Yeah, that's exciting. That'll be our first opportunity to meet in actual person.
- [03:42]
A
In real person. Yeah. In real life. Yeah, yeah, yeah. For anyone listening, Sean and I have never actually met. Met in person. So, yeah, this is going to be cool. And you're coming to Singapore at some point as well.
- [03:52]
B
We have two opportunities.
- [03:53]
A
There we go. Yeah. Whether we'll do an in person episode, that's a different thing because actually recording in person means like a whole studio setup, I think, or something. Whereas people are surprised to hear I've
- [04:06]
B
done a few in person. There's a lot more technical complexity with doing in person than just recording on your computer remotely.
- [04:13]
A
Exactly. Yeah. I've had a couple of guests on SE Daily just say, oh, can we go to a studio? And they're like, well, we don't have a studio. SE Daily is a remote podcast. That's how we do it. Right. Moving on to the headlines. So some of you that do listen to this SED news a lot. We have talked about ARM in the past, but more so the lack of arm. But I think the headline here is ARM is back in a big way. And this all kind of focuses around CPUs, because the headline stories of any of this compute inference, the story's really been around GPUs. And we did talk a little bit about TPUs, which are Google's derivative of that, but we are really going back to CPUs. So. Yeah, Sean, what did you pick up on this one?
- [04:59]
B
I mean, it's like everything in text, like the old thing is now the new thing again. It's always cyclical. I think part of it, the reason we have GPUs, we have GPUs and now CPUs are back. It's just you need a lot of compute to run all this stuff. If you're running models, you want probably GPUs, TPUs to be doing sort of the matrix multiplication there. They're really good at that. But CPUs are also good at certain things. You know, they're good at logic and branching and task switching. And I think, given now that people are running these agents on their desktop and not just running one, like running multiple agents. I talked to a startup that I worked with recently and I was like, how many instances of Clotter you're running simultaneously? And they're like, well, I start to lose track of things when I get up to like six. So that's a lot of compute that you're running locally. And I think essentially that's what's kind of driving this, is that people are running these agents locally. I don't know that that was something that we necessarily foresaw in the industry a year or two ago, that so many people would be kind of running these agent experiences on their local computer versus the cloud or something like that, where maybe you're using some other type of compute. But if you're going to be running these locally for things like openclaw or some other version of Claw that now exists, or you're doing it with the agentic engineering tools that are out there, you're just going to be consuming up a lot of CPU and a lot of memory. So there's now a lot of, I don't know, appetite, I guess, in the industry to find more of this compute.
- [06:22]
A
Yeah, and I think one of the interesting pieces here is that ARM is actually going into more of the actually manufacturing their own chips. Not fabbing exactly, because you have Fab being effectively the sort of manufacturing plants that can do this. And those are incredibly. There's only three really players that have those. I believe tsmc, the Taiwanese is the main player. Intel do have that capability and Samsung is actually the other huge one that has that capability. Not entirely clear who's fabbing these for arm, but the difference being that ARM traditionally was supplying the designs of chips like that's their whole thing. Virtually every chip that is created has some design piece in it that comes from arm and so they basically take a royalty, if you want to call it that, off almost every single chip that is created in the world. But they hadn't really gone full on into hey, this is like an ARM chip with an ARM logo on it for example. And this is kind of where they're going. And yeah, as you called out Sean, it's the rise of agentic tasks and it really has just accelerated as we touched on I think in the last SED news and in the last two, three months that suddenly everyone's doing agentic stuff, not just engineers. Anyone seems to be quite interested now in agent getting an OpenCloud running and leaving it on overnight on their Mac Mini or something. So there's clearly just a lot of appetite now for having these independent environments that have their own CPU to run a specific agent. And I think that's kind of what we're seeing here.
- [08:00]
B
Yeah, and I think that's only going to continue. These companies, whether Anthropic or others, are verticalizing a lot of their success around Claude Code, Claude Desktop to be geared towards finance and geared towards lawyers and whatever profession that you have. So every sort of white collar job eventually is going to have people who are running these things in some sort of environment. So you need compute, you probably need some sort of containers and security perimeters around this. So there's not just, there's I guess like a ton of people startups all the way to large companies that are really trying to meet the needs of the market right now. And it's something that like everything now when it comes to AI, it's exponential growth. It's not this kind of like nicely growing linear thing. It's like as soon as something takes off it's like extremely viral and companies have to move really fast to essentially meet that demand.
- [08:51]
A
Yeah, I mean Nvidia are in the space as well, they offer CPU racks, but I guess they're still known, I guess on the GPU side. But yeah, it's kind of, I would say like quote, great to see CPUs coming back. I think the fact that they've powered so much to this point in time from a technology standpoint, and then everyone kind of thought, oh, they're like yesterday's technology almost. They're incredibly powerful, but for specific tasks. And you don't always need an incredibly expensive GPU running every single task, as I think a lot of people have decided just for the sake of they can afford it, that's what they're going to do, but it isn't actually needed.
- [09:24]
B
Yeah, when I did an SED episode, unfortunately I can't remember the person's name, but he was a Turing Award winner. In supercomputing. One of the topics of conversation that we had was how the industry's focus so much on workloads catered towards essentially the math behind model crunching and inference has led to essentially poor performance in supercomputers for certain types of tasks. So there is always a trade off. If you overly specialize in something, GPUs are really good at a certain type of math. TPUs are even more biased towards being really good at that particular math. You're also giving up something that might be important for other types of logic or compute that doesn't essentially fit the profile that you're going to run on one of those other pieces of hardware like a gpu. And when you're running agents, it's not like it's just model crunching. There's all kinds of other things that they're doing where you want to be able to ideally use the right compute, depending on what the profile of the task is exactly.
- [10:23]
A
And it is just the volume here. There isn't really a sort of realistic world for an average person at the moment where every agent has its own gpu. So even for most people in tech, the numbers don't add up, if that's what it takes. So if you want the number of agents that you think would get your tasks done, you're probably looking at CPUs to run most of those logic tasks. So, yeah, very cool to see ARM back in the fold. And yeah, I read a book about ARM kind of just before, I think it was just before they IPO'd or something like that. And I just, I really fully at that point then understood like, wow, they have just behind the scenes powered so much of what we take for granted with all these designs, but never really got the recognition for it. So it's kind of, yeah, it's fun to see them actually stepping into branding their own CPUs as well, so exciting. Moving on to this does sort of hit the main tech headlines in the sense like this was a lot of reporting by TechCrunch on this one and I think especially anyone in software engineering has maybe seen it on Hacker news as well. But I do think this kind of really crosses into main news in terms of our world. The Light LLM hack. Now this was a takeover of a dependency, effectively that then extracted the credentials of many services. So Light LLM basically gave nice API access to a multitude of LLMs and helped with things like metering and so on and so forth. So it was a pretty used tool by so many developers out there. So that kind of makes it unfortunately incredibly ripe as a target for someone to think about what can we do if we can get the credentials to that? And I think it's just the focus here. Lightlm themselves were a X Y combinator. And then, yeah, sort of the kicker here is the fact that another Y combinator company, Delve, they were already in a bit of hot water for potentially fabricating. So they're a compliance startup. SOC2 reports giving you go through the checklists and you get the SoC2 is still a funny industry where it's actually accountants that do the final stamp on things. That's a whole other story. But they apparently they're auditors as well. Yeah, exactly, they're your auditors. And it still seems kind of strange that you've got effectively a financial company saying that your security of your technology is up to scratch, but that's how that system was put together. And there's been allegations that Delve have been fabricating the sort of the rubber stamping of these reports. Unfortunately, Delve had given one of their reports with a clean bill of health to Light ALM before this happened. And I think that people are pains to point out getting SOC2 does not mean they have checked every single process that is going on in the developer's workflow within Light LLM, for example. But it should be a pretty good indicator that they're following best practice. And well, something kind of wasn't going by best practice here and it's had a catastrophic effect on a lot of developers. But yeah, that's I guess this sort of slight security minded focus from me. But what did you make of this one, Sean?
- [13:34]
B
Yeah, I mean I think it's just another example of where compliance isn't necessarily security compliance is really a series of kind of check marks where you're saying, yes, we are following the best practices, but every company pretty much that has some sort of data breach, has some sort of hack, is incompliant. They pretty much all have like SOC 2 or whatever, various ISOs, like whatever sort of security check marks that they need. And I think in a lot of ways, like compliance is really about insurance, while security is actually about trying to stop the attacks. So there's a very big difference between, I would say the companies are really secure by default and they take a hard stance on this versus people who are, hey, we have to make sure that we have these badges so that when we're going through procurement with a vendor, they're not going to immediately just say no. I also think the interesting hearing here is where a lot of times when we think about hacks historically or breaches, it's like, what is it that people are trying to go after? Well, they might be trying to go after someone's password, they might be going after credit card information, banking information, personal information for like identity theft. And here I think there's now this like new industry that's going to be the value is, hey, I want your OpenAI API key or I want your anthropic API key. And then not only can they run up bills on you, but they could be leveraging those models to be doing something malicious against a bunch of other people too. So they can essentially spin these things up using your precious tokens, run up a big bill for you, while also using that thing to attack somebody else.
- [15:05]
A
Yeah, so this was discovered by a guy called Callum McMahon of Future Search, which is a company that offers AI agents, apparently for web research. But he documented how this all unfolded, basically ended up with his machine being shut down. So that's a pretty clear indicator that something has really gone wrong here in terms of how deep this sort of hack has gone. It's something we've covered on SE daily with guests obviously of companies that help to mitigate this. The one that comes to mind is Wiz. We had Rami McCarthy from Wiz and we actually covered a case that he'd worked on that is exactly this. It's like a dependency has a sort of leak within the maintenance of it and that just cascades through so many other packages. And this one particularly just the number of developers right now using Light LLM is just shown to be such a problem.
- [15:58]
B
I think these kind of like dependency supply chain attacks are hard to catch and it's easy thing for attackers to kind of exploit. And then in a world. And we're going to talk more about this during sort of the main topic but in a world where we're using AI more and more to generate code, and maybe not always looking at that code that closely and is picking up packages in order to do some type of work on our behalf, we might not have as clear a handle on what those dependencies are. We're kind of just looking initially at the outcomes of that. Does it pass the tests? Does it kind of look correct? If it's generating some sort of UI based on what it is that we're trying to accomplish, we commit that thing and then we get it running in production and then boom, we have some sort of problem.
- [16:43]
A
Yeah. So the next headline here, I guess again, this might have crossed more slightly into the hacker news from a headline perspective, but I think it was interesting that open code really made a lot of noise in the community here. And this really goes to the fact that agentic coding has just taken off, but most of the time we were talking about either cloud code or we were talking about codecs from effectively chatgpt. And that means you're buying into again, one of the big names. And here was a product that's able to say, hey, we're actually fully open source. You've got access to local models, you've got access to free models, or if you really want, you can hook it back up to Opus or something like that. But I think that's just the amount of opinions that were then given about it is kind of really interesting.
- [17:31]
B
Yeah, I think we're going to, of course, see more and more of these open source alternatives. In a lot of ways it reminds me of if you looked at the ide market maybe 15, 20 years ago, it was very similar to where open source IDEs started cropping up as alternatives to the mainstream ones. And the thing that I always thought about, or since people started even with GitHub Copilot, was I think we've always gotten to a place where developers don't really like paying for their tools. And then you end up with open source alternatives. And then with the IDE market in particular, the IDE very much became a commodity. And there are certain ides that survive that and people pay for. But predominantly, I would say most became open source or were given away. Even things like Visual Studio, that historically had been a big a moneymaker for Microsoft. So today people are able to monetize these agentic engineering tools because they are incredibly valuable. If someone took away my quad code, I'd be very upset about that. So there is a lot of value there. But as you have more and more of these open source equivalents that are maybe even if they're not as good, they might be as good. But even if they're not as good, maybe they're 80% as good. People are happy to use something that's 80% as good if they don't have to pay X dollars a month to use some other tool. So then you get into a world where people are willing to pay for the tokens, but people aren't willing to essentially pay for subscriptions at scale to have those tools. So if you want to monetize that market, how do you create a big enough value and a big enough moat around what it is that you're providing so that people don't just go to the open source equivalent.
- [19:07]
A
Yeah. And it was kind of interesting just to look at some of the conversation around it in Hacker News as well. And I guess what maybe even I hadn't sort of fully appreciated is the fact that Claude code, okay, it runs in your terminal, but it is actually an electron app. I didn't even realize this. And that's how you kind of achieve all these amazing visual niceties effectively. Yes, it's in a terminal, but there's a ton of stuff going on from a rendering perspective, like terminals are being hacked a bit in this way. I love the visuals, but then I was kind of starting to think how is this even possible inside a terminal? Codecs apparently is done in rust and people say that that is a tangible speed difference. And one of the things, yeah, and I think one of the things that has been questioned about open code is the fact that basically you're using up a gig of RAM basically for a terminal, and that it's a pretty chunky typescript code base that's running this thing. So this is it when you're designing these kind of tools for developers. Well, guess what, you're going to have some pretty, I think, spicy opinions on how did you put this tool together in the first place. So I think that's going to be an interesting adoption data point that they have to think about in mobile application security. Good enough is a risk. Guard Square uses advanced multi layered code hardening techniques and automated runtime application self protection and mobile application security testing combined with real time threat monitoring to deliver the highest level of mobile app security. Discover how Guard Square brings all these together to provide mobile app security for your Android and iOS apps without compromise at www.guardsquare.com. today's episode of Software Engineering Daily is brought to you by Unblocked your coding agents have access to your code base. Maybe you've even connected other tools via mcps. But access doesn't mean context. Agents can't reason across mcps. They don't know your architectural decisions, your team's patterns, or why the API was shaped the way it is. So agents look in the wrong place and deliver bad outputs. Then you spend time correcting turn after turn. Unblocked is the context layer your agents are missing. It synthesizes your PRs, docs, Slack and tickets into organizational context that agents actually understand. So they make better plans, write higher quality code, use fewer tokens and require fewer correction loops. If you're running Claude, code, cursor or any agentic workflow, Unblocked is worth a look. Get a free three week trial at Get Unblocked.
- [21:48]
B
You know, there's a lot of these tools now available. You have Claude, you have the mainstream sort of commercial tools from OpenAI, from Anthropic and from others. You have the open source equivalents like you have Klein, you have open Code. There's a whole plethora of these. How many can the industry really support? I feel like sometimes one of these tools crops up, kind of catches fire for a period of time, gets a lot of GitHub stars and love and people start using it and then it kind of tails off as the new hotness hits the market as well. So I don't know that that's sustainable in the long. At some point, I think some of this stuff starts to level out and they'll emerge like a couple of main players that dominate the market. And maybe there's a couple alternatives, but I don't think you can have like 100 alternatives.
- [22:30]
A
Yeah, and just to round out the headlines, this sort of, I guess, broke cover maybe a couple weeks ago, but I think it just sort of is useful to touch on a slightly wider story that we're seeing play out, which is how OpenAI and Anthropic are kind of how are they playing their markets now? And there is obviously a lot of noise around how the Pentagon, they wanted to, the Department of War or justice or whatever they're called these days, wanted Anthropic to allow CLAUDE for what they call all lawful purposes. And that does include things like surveillance and autonomous weapons. And Anthropic refused and they didn't get the contract. And then apparently OpenAI quotes swooped in and signed a deal. And there was a lot of, I think, user backlash around this. And it did make Anthropic look like the real human winner there. If you want to use one of these technologies, why would you use the one that really wanted to get in with the government, the U.S. government? And if we kind of then just play that through to where have we seen OpenAI versus sort of anthropic? Where are they positioning themselves? It does to me at least look like OpenAI started to becoming this kind of super app type construct, which we've touched on before, Sean, in sort of other deeper topics. And Anthropic, Claude, they're really going full enterprise and say in a positive way, but they're really going and saying we can be the enterprise tools that you need. There's all kind of, if you loop this back to the whole Pentagon thing. Well, a lot of companies really don't want to be associated with a company that they think is aiding the government with, say, surveillance, or as we speak, the Iran conflict is on right now, and we don't know how the technology is being used there. So just kind of staying away from that, I think, is what companies are looking at. And it's just a very interesting development, I think, in this sort of landscape of the two companies.
- [24:26]
B
Yeah, absolutely. I mean, I think that it ends up kind of highlighting, I think, something where those two companies are already somewhat have opposing views of what is the right thing to focus on when you're sort of responsible for building these models. And even this. Dario and some of the other founders at Anthropic were OpenAI at one point, and they kind of split off because at least the story goes that they didn't agree essentially with the philosophy that OpenAI was taking towards building models and releasing models. They wanted to make sure that as a company that before they released the model, it was really well tested and really security and trust was sort of the number one value. So they splintered off, built Anthropic. And I think this whole thing that's happening politically has been probably some of the best marketing that's ever happened to Anthropic. You saw people leaving or canceling their OpenAI subscriptions. I think even under the best of circumstances, like with politics, people, companies, individuals might not be that comfortable with a company supporting something like surveillance of individuals or automated warfare or anything like that. But on top of that, we're also in a place in the United States where politically things are very polarizing right now, and there's a large chunk of the country that doesn't want to be associated with the government at all. So this is a very, very polarizing issue. And Anthropic's kind of chosen to take one stance on this. OpenAI has taken a different one and as a result I think it just highlights the differences between the two companies even more.
- [25:59]
A
Yeah, and it's not like we've seen things like this probably appear in the past where a company potentially anchors itself a bit closer to a government or not. I'm not going to bring up any specific examples, but what I will say is in the past, okay, would one have refused to use. I'm just taking pure examples that might not be true here, but would I have refused to use Google if I knew that they were aligned with a certain government? I'm not sure because, okay, it's a lot of your search history and that kind of thing. But now that we're Talking about what LLMs are used for, well, quite often people are divulging a lot of their personal information beyond just, oh, this is what I purchase or this is where I'm located right now, which is personal information, of course. But people using LLMs, they're kind of writing about their deepest, darkest thoughts. And in companies especially, people are just literally hooking up, whether through official integrations or full on just pasting in Slack conversations and documents and that's it. Where does that information end up then ultimately? And I think that's why this is such a concern for companies especially.
- [27:04]
B
Yeah, absolutely. There's something, I think more personal about chat than necessarily search. I think people have kind of taught themselves that, okay, well maybe I shouldn't put my Social Security number into Google search. I don't know that people think of that necessarily with chat as much or they paste documents in that are incredibly sensitive. Not really thinking that through because of the value. There's just so much value with using these tools that people kind of throw out maybe their reasoning skills sometimes with like, should I actually be sharing this? What is the consequences of sharing this information?
- [27:38]
A
Yeah, I've definitely noticed that people that previously would have said, I will never divulge certain information to a tech company. And then the next day, oh, I've had this amazing conversation with Claude and told them all these things and I don't comment on it. I just think that's really, I'm just observing. It's very interesting that this is someone, for example, that I know would not have done this in the past, but actually just they've felt some trust there. That's a whole topic for a different day. But I think it just sort of underscores who you give your information to, especially between the, you can call it like the two main players here, Anthropic and OpenAI, as soon as one then anchors themselves to a entity that people disagree with, that adds a whole extra layer. So we'll watch how this plays out, I guess, and yeah, we'll sort of see does this affect OpenAI enough that they have to walk anything back or are they kind of going more like Palantir style? They actually think there's too much value to be had by being very close to the government and helping them achieve things. I guess we'll see how that happens.
- [28:37]
B
Yeah. And I think from a consumer standpoint, a lot of times when there's so much value behind whatever the product is, consumers end up turning off a blind eye to maybe things that they wouldn't normally agree with. You can take any retailer, major retailer in the world that's probably manufacturing goods in developing countries where maybe the standards of work are not necessarily great. People kind of sort of know that, but they also like their inexpensive T shirts or whatever and they're kind of willing to turn that, that part of their brain off because they're focused on their own personal outcomes. And I think you could potentially see something similar with, even if you don't, in conversation, agree with what OpenAI or any other company is doing, there's so much value behind their tools that you kind of turn it off when it comes to your own personal objectives.
- [29:27]
A
Yeah. So it's that time. Within SED News, we're going to move to the sort of main topic deep dive and today we are looking at effectively writing code versus shipping code and especially the emphasis on shipping code. How does this look in the age, if you want to call that of LLMs now that probably at least 50% of developers in a given company are going to be using LLMs to write code. What are the kind of maybe cascading effects on how that code actually then ends up, to be frank on main branch, what does main branch look like from a speed perspective and actually getting things merged in. And we kind of anchored this around. There was a Circle CI as a company that many people know in the CI CD space and they did a 2026 state of software delivery report. So that was just back in February. We're recording this just at the end of March. They said that they analyzed. This is going to be quite data heavy, this main topic. So get ready for some numbers because thinking about when we talk about writing code versus shipping code, it's basically old numbers like lines of code and how many merges were made and so on and so forth. So they said that they analyzed 28 million CI CD workflows across 22,000 orgs in 149 countries. That's a pretty good spread. This is the interesting anchor points here that the daily workflow throughput was up 59%. But it's kind of misleading that the top 5% of teams nearly doubled the throughput while the median team increased only 4%. So that basically suggests that super high performing teams could super accelerate their throughput. That is things that got merged into main again just to kind of bring it back there. They could double their throughput, the top 5% of teams, but the median just call it like the average. I know average median is different but when people think about it that to only increase 4% on throughput for most, for just call the majority. That's pretty interesting because most developers are saying, oh I'm so productive, I can do so much more per day now if I use an LLM. But if actually from a team perspective and a shipping perspective, the majority are actually only increasing by 4% that looks way off what we might have expected I think from this, you know, there
- [31:51]
B
could be a couple of things going on. One is that, you know, not everyone has reached the level of the like the most high performing teams. Adopting some of these tools is still relatively new to many parts of the industry. So you're going to have people who are kind of trailing behind. Maybe they're kind of as a company, maybe just you know, dabbling or this is like fairly new efforts. So they're kind of getting used to how do we get value out of this. But I think the other thing too you could say is like, well maybe the teams that are shipping these like you know, top performers, maybe they're shipping a lot, but are they shipping also a lot of bugs and they're just okay with that, whereas other people are trying to control that. So there's a bunch of stuff that it would be nice to kind of be able to dig into the details of. Why is it that some people are at 4% increase throughput whereas you have some people who are double this? And there could be many, many different reasons for that to be the case. The other thing too is that just because you speed up writing of code doesn't mean you speed up the entire software development life cycle. You have this software development lifecycle of just a very simplistic one of discovery requirements, coding, testing, putting it into production and then running that thing in a loop. Well, if you speed up that middle part of writing the code, it doesn't necessarily mean all the other parts of it has sped up as well. So those can become choke points in the company. And on top of that, if you speed up the lines of code, those choke points might become even slower. Because the thing like, let's say it's your security team that has to review these things. The fact that they were somewhat slow in the past was maybe okay because the part of writing code was really, really slow in comparison, but now they suddenly have like a thousand times more things to do. So what happens as a company, either you cut corners and you start releasing products that don't necessarily go through that full security review, which leads to negative consequences, or you slow down the entire release cycle so that you only get 4% increased throughput because the poor security team is completely slammed and overwhelmed. It's the same thing. Even with reviewing PRs, if we're generating more code, that means there's more PRs to review, which means that that has to be paid for from somewhere. So either you have your engineers spend all their time essentially reviewing PRs, which slows down, of course, things getting into production, or you leverage LLMs to also review the PRs, which then also potentially has certain consequences where less and less people within the company really sort of deeply understand what is happening in the code base, what the dependencies are, and then you end up with some sort of issue on production. And then the people who are debugging it are also not that familiar with it because AI wrote it. There's all these kind of consequences of what's happening in the industry right now. Whether it's like, hey, we're not really seeing the full throughput value of the LLM because we're generating code a thousand times faster, but these other things are getting completely bombarded and slowing things down. Or we're generating code a thousand times faster and we're pushing it out of production, and then we're also generating bugs a thousand times faster.
- [34:58]
A
Yeah, the kind of stat that does really make this, you can really visualize it, is that feature branch, like this report that came from Circularci. They saw the feature branches, the throughput was up 15%. So, okay, great, 15%. Generally, more feature branches are being created, but the main branch throughput itself was down 7%. So basically. But actually, sorry, that's throughput. And the actual creation of branches, feature branches were up 50%, whilst the main branch, again only like, sitting only at 1%. So it basically just means that a ton of code is being generated and a very Very, very small fraction of that code is ever making it onto main branch. And I think to kind of go exactly to what you were just saying there, Sean, that middle piece, the human in the loop there, is still obviously like the biggest question mark. You could call it a bottleneck. But I think most companies are saying it's still the most critical thing that they believe needs to be there. It's all very well generating a ton more code or a lot faster at least, and having all these supposed features or so on all queued up and waiting to be potentially merged in. But the actual review piece, well, that's where anyone really is going to be saying, well, hey, we're not comfortable with these just being whisked through by an LLM that sort of knows how we do things. And I think that's the piece that's going to take a lot longer to really kind of make it through to say, production. Production. Apparently there's an example from the new stack back in March. We're still in March, so coming out in April, but we are.
- [36:42]
B
Time is a flat circle, Gregor.
- [36:44]
A
Yeah, well, it is in AI terms, yes, or AI age. But VS codes, they've actually moved to weekly releases after 10 years of monthly. And they say AI is actually making this possible, but they say that only works because they say. I mean, Obviously, I think VSCodes specifically probably have reasons that they at least want to push the idea that this is all possible, but they say it only works because they have strong automated checks and a team that's invested in this review pipeline. So, yeah, I think it's just. What does it actually look like to try and get your team around this paradigm, I think is probably what a lot of people are wondering about.
- [37:24]
B
Yeah, I mean, I think that the reality, at least at the moment is organizationally you have to probably change because if suddenly the resources that are scarce is not necessarily the production of code, but the resources that scarce are essentially verifying the code is correct. Adi Osmani has written about this. Said generation is not in the bottleneck anymore. Verification is. So if it's all about PR reviews or in the world of infrastructure, the core piece that a lot of times really matters is networking, security, how you do authentication. And if you mess that up, basically you might destroy the core infrastructure of thousands of different customers. You can't mess that piece up or it's going to be dramatically negative consequences to your business. So that's where you really need to take care. And maybe you have to deploy more resources there and you could kind of cut back resources in other places where suddenly you don't need five people to maybe generate as much code, but you do need five people to verify the code is correct. So it's almost like we're shifting the problem. Where the problem historically has been it takes a long time to write the code. That's been the sort of hard, long cycle. And then now, I don't know that we've really sped up everything. Like I was saying, we've kind of shifted where the problem occurs essentially. I think the really negative part of this is that in the sort of media we focus so much on these headlines of like such and such, a company replaced a team of engineers with one engineer and six open clauses or whatever it is, or people are going from idea to, to app with one prompt and built an application in 20 minutes. And I think the thing that we're confusing is that when the demo comes together in afternoon, I think it ends up recalibrating what the stakeholders think possible. Executives who watch some AI generated prototype naturally ask, hey, if we built this in a day, why does production take six months? And it's this whole 9010 rule that has always existed in software where first 90% of the work takes 10% of the time and the last 10% takes the other 90% of the time. Because there's all this other stuff that has to happen around productionization. And we're really confusing prompt to prototype with prompt to production. And I think that is going to have massive consequences of the industry. And I know engineering teams are under tremendous pressure right now to be shipping at the speed of essentially prototyping and you either end up not being able to do that, which could be negative for your career, or you end up in a situation where you bypass the normal checks and balances and then that's going to, I think, lead to more and more of these outages and failures. I mean, even if you just statistically take assume that with human generated lines of code to the number of errors that you have, if that statistically stays the same for AI generated code, then if you're generating more code, you're going to have more errors and you're going to have more production outages. So I don't think it's a particularly bold statement to say that we can expect more outages from companies over the next year easily because of the fact that if companies are in the place where they are actually able to put this into production faster and there is all this pressure to do that, we're going to end up just Generating more code that generates more bugs at some point.
- [40:49]
A
Yeah, as you touched on this, it's going to be a complete. It's sort of a culture competition at this point because the company that already wasn't putting a ton of thought into the pre writing code and then the post writing code piece, well, they never saw that anyway. So like well, awesome. We can just accelerate the bit that we thought about most which was just generate code and generate stuff and ship it. And yeah, I think they're starting to get sort of a bad surprise in many areas when things are don't suddenly accelerate 10x because the problems have accelerated 10x and it's the companies that already have these literally from writing whatever people call them, like PRDs or RFCs before something gets shipped again if you augment that process with LLMs. Well actually that's very interesting because a ton more thought and discussion can happen around something before it's even thought to be getting code written against it. And then the code being written against it potentially gets sort of accelerated. But I can't say that I think the best features are getting accelerated by LLMs. It's actually, I would almost argue it is maybe that first piece where the LLM has done a ton more thinking behind the whys and the hows that everyone can then read over and comment on and then a classic pizza team works on the feature itself and that's still exactly what should be happening. Not to make me send anti LLM on code because I'm very much pro. It's just interesting to see where does bringing LLM into the loop actually affect the final outcome and the quality of the outcome, including security and robustness and all these kind of things.
- [42:34]
B
Yeah, I agree. I think that I certainly don't want to come out sounding like I'm anti AI generated code. That's absolutely not the case. I think these tools are tremendously valuable and as I said earlier, I've be very upset if Claude code went away very dependent on these things at this point. But I think that the problem here with sort of sensationalizing the fact that you can go from prompt to building a compelling demo and then sort of confusing what that really means hides some of the real value here. I think one of the things that this gives us is very interesting and you kind of touched on it there where in many ways software has become a thinking tool because the cost of essentially generating that idea is so low now. So as a product manager like myself, I could spin up a working demo of an idea instead of writing A one pager and then hoping my stakeholders can kind of visualize it. The demo isn't the product, but is a better way to have a conversation essentially about the product that we want to build. And in many ways I think we're entering this era of where AI is kind of making software ephemeral. And that's a real shift in how we can do work. Because anybody, not just people who are technically adept, suddenly you have people who with a little bit of training can have the ability to convey their ideas through software. And I think that's very, very interesting. Because if software can only be created by a small set of people in the world, you're of course going to have certain biases that come with that or certain worldview. If suddenly you open that up where it doesn't mean that you're building production software, but if anybody can kind of prototype something, then it's a way, I think for a completely generation or essentially a completely different set of people to be able to convey their ideas and software. And I think that could lead to some really, really interesting outcomes that we haven't seen before. And that's always the case when you kind of bring new worldviews, new ideas, new cultures to something that didn't have it before. So I'm excited about that and I think that there's tremendous value here. I just think that a lot of the real value gets kind of lost in the conversation. And by overemphasizing the idea that prompt leads to faster productionization, it's going to have negative consequences that companies don't really necessarily foresee right now. And it's hurting engineering in general by sort of misleading where the real value is right now.
- [45:03]
A
Yeah, so kind of the, I guess the sort of TLDR here is just effectively that your sort of validation layer, if you think that LLM coding, like the point of it basically is to increase throughput generally, and by throughput here we would define that at this point is as merging domain in a state that meets the bar of quality that you as a company think is acceptable, then the validation layer effectively needs to keep pace with that. And at least, yeah, I mean where I sort of sit, I'm not talking specifically about where I work, but you can look at a bunch of code bases, open source, et cetera. It does not look so far like that is quite keeping pace if that's your kind of metric. Because rightfully probably at this point the human in the loop is still very much there and still a lot of post discussion happens and A lot of the volume of code needs to then be reviewed by a human. And that's a very labor intensive task because I get the impression that a lot of people aren't then comfortable saying, well, I'll just use an LLM to review the code that was generated by an LLM. I'm like, okay, sure, I know that people use models to rank outputs of models, but this is precisely the piece. I don't think anyone's going to still have a job at the end of the day if they told their engineering manager, well, the code that was written by an LLM. All I did was have it reviewed by another LLM and then I pressed merge, like in professional software production for big companies, for example, that we work at that just is not going to have you a job at the end of the day. If something goes wrong, it will just be you on the end of the line saying like, well, that was not the way to achieve that.
- [46:39]
B
Yeah, I mean, we both work for companies where if we mess up something in our core infrastructure, that means we mess up the core infrastructure of potentially thousands of customers.
- [46:51]
A
Oh yeah, six figures of people could get messed up easy by one problem.
- [46:55]
B
Exactly. So there's massive, massively negative consequences to something like this. I think part of it also is companies kind of have to think about. If you want to accelerate things, are there certain parts of the stack that you could accelerate? Are there places where a bug is more forgiving than other places? Maybe it's not networking, but could it be part of the UI layer where you can move faster? Because if you end up messing up the UI some way you can fix it immediately, push out a change. It's maybe less catastrophic depending on what the company provides. I think you kind of have to think of this as layers of an onion. Which layers of that onion do you need to be really, really scrutinized and make sure are up to whatever your par is? And then are there other places where you can be a little looser on that and the consequences are not going to be as catastrophic?
- [47:49]
A
I think that's a great place to leave it. So, yeah, just hopefully some interesting thoughts there on writing code versus actually shipping code. You've probably got a few anecdotes, listeners thinking about how you work in your own companies now and just what does that look like in your own flows? And we're always happy to hear about those as well. Moving on to our final piece of the show. Usually our kind of favorite piece of the show. This is hacker news, but nothing too serious. Just interesting things that have popped up on Hacker News. I think I could scan to see what Sean's is and I want him to go first because I definitely saw this as well and I was like, I guess we're going to cover this because spoiler, it's to do with Doom. So, you know, it's going to be Doom on something. What is it this month?
- [48:36]
B
Yeah, it's so funny to me. Probably every week, certainly every month, there's always going to be a Doom on something project on Hacker News that, you know, catches our eye. But it's also funny to me how like Doom has just become that thing that everybody wants to like jam into anything. We're going to run Doom Watch. We're going to. You put it in your wifi enabled earbud. And now this is Doom on over DNS because why not? So really, really fun project. I think then one was the person who posted it. There's a GitHub you can take a look at. But essentially the project compresses the entirety of the shareware, Doom, splits it into nearly 2,000 DNS text records across a single cloudflare zone, and then plays it back at runtime using Nothing but a PowerShell script and public DNS queries. It's just like amazingly absurd, but incredibly awesome.
- [49:28]
A
I do think, despite the Doom in typescript types, which was an incredible feat and we did have an episode on that with one of our other hosts, I got to say, this, to me is creatively just one step above, because I don't think anyone thought that writing software via DNS was a thing or could be a thing. And as usual, Doom, I guess. Why is it always Doom? I mean, it's just a kind of meme developer meme at this point. But the fact that it's this very tangible outcome that as soon as someone's like, well, it's Doom, it's not just saying, oh, I created like a calculator app from DNS. It's Doom, a game that's insane. So, yeah, that's very.
- [50:11]
B
Yeah, I'm sure the creators of Doom back in the day had no idea. They had no idea that they were going to become like the. It's almost like some sort of benchmark standard at this point where it's like, can you get Doom to run on X device or whatever it is.
- [50:24]
A
Yeah, the two Johns, I read that book Masters of Doom. That's a great book if you want to ever hear about how Doom actually came to be. But nothing to do with all the funny Ports we've been talking about. But yeah, on my side, a couple of sort of just interesting ones. One was why this is called why so many control rooms were Seafoam green. So I often do bring up some of these, maybe more designy ones that hit Hacker News because it seems like Hacker News has this amazing community of people that can sort of dig into why things were designed a certain way, whether it's pure colors or so on and so forth. So this was posted by user Amori Meltzer. Thank you for that. And the TLDR is Seafoam green is apparently when we talk about seafoam green, we're talking like a sort of very light green color. Mossy light, very light moss in Scotland right now. I see it everywhere, but think of a very light pale green and it's on walls in these sort of bunkers of control rooms. And why is that the case? Well, the TLDR is there was a chap by the name of Faber Biron who was somebody at the Art Institute of the University of Chicago. And when he was in New York he actually became a color consultant and he helped people understand. Well, if you paint your walls this color in a certain restaurant or a butcher, for example, that might drive more sales of steak because the meat looks redder against a certain colour, for example. He was then hired by the government to, during World War II, to create palettes for different buildings effectively to help people just understand, oh, this area is a danger zone. So fire red was used for all emergency stop buttons, which I think we would think of as pretty obvious these days. But that was not obvious before some thought was put into it. Anyway, light green, that was simply used to reduce visual fatigue. Apparently it's seen as a color of it. You can sit in a bunker with all these switches and dials all day and that is apparently the color that reduces fatigue on the eyes. So given that you've got to be staring at back then, certainly staring at all these other machines. So very just interesting, funny bit of trivia there.
- [52:37]
B
I've heard sometimes certain fast food chains choose a particular color in order to encourage people to not stick around because they want to turn over the seats. I don't know how much truth there is to it, but there's definitely something where certain colors I think do affect people in different ways.
- [52:54]
A
The one that I never understood and it's probably why I have not flown this airline for maybe I can't even remember since I was a teenager, literally. Ryanair. I'm sure some people know about Ryanair it's a very low cost airline in Europe. But they used to put a very bright yellow plastic thing on the top of their seats and it was just like headache inducing. And the problem is you can't leave the plane. You're on the plane, you can't get off for the next two hours or whatever. So I have no idea why this was. They have a very controversial CEO. But yeah, you were just talking about wanting to make people leave your establishment. Well, that makes sort of business sense to me. But why trap people in a plane where they probably want to get out? That does not make sense to me. But what else did you pick up, Sean?
- [53:37]
B
Well, so the other one that caught my eye here was a guy who basically took Tesla Model 3 like crash car components in order to assemble the Tesla Model 3 computer on his desk. And this was motivated by Tesla's like Bug Bounty program. But he had to like salvage a bunch of this hardware from these various parts. The article kind of goes into detail about like how he did this. So the, I guess the car computer has two main parts. There's a media control unit which is like the mcu and there's an autopilot computer or ap. Then you need like power supply, you need a touchscreen. But I think the thing that he ran into the most challenges around was just kind of like connecting everything because the actual connectors in the car is kind of like this massive cables looking at the process.
- [54:27]
A
Crazy amount of cables.
- [54:29]
B
Yeah. So then he had to figure out how can he actually connect the MCU to the other devices in some way where he could provide the right cabling. Because essentially it's hard to find that part because it's either been destroyed or it's just this mass of cable that's not super useful.
- [54:48]
A
Yeah, that does feel quite deep end. Bug Bounty. Okay. Has provided a reason for that. But yeah, still, that is like, you got to be really into this kind of stuff to pick through how on earth this is going to all fit together. Just a quick one to round out. I think it's an interesting. Yeah, the Mac Pro has been discontinued by Apple. No, yeah, I know. I can still remember, yeah. Going into the Apple Store, you know, in like 2012 maybe in New York. And like the Mac Pro is like this like beautiful aluminium box. And it's like that was if you were any kind of like serious industry, especially you know, video or something. Like if you had a Mac Pro, that meant you were a professional on this. Like. But apparently, yeah, Mac Studio is kind of where they're headed Mac Studio. I actually just looked this up. I wasn't fully aware of it, but Mac Studio kind of looks like a Mac Mini, but it's just got, you know, beefy internals and it looked like their kind of product lineup was a bit muddled by this point, but they had the Mac Studio and they still had the Mac Pro. They've decided to officially kill the Mac Pro and just focus on the Mac Studio line. So probably quite a sensible move, but yeah, a little bit sad for some of us that still remember all the. That was the sort of. If you ever wandered in someone's office and they had one, it was kind of cool to just use it and know that this was the most powerful Mac on the planet at that point. I guess briefly looking ahead, nothing you're thinking might pop up when we're talking next.
- [56:17]
B
I kind of already stated my point of view on prediction, but I think we're probably looking at more outages
- [56:24]
A
with this extra code and validation problems. Yeah, for sure.
- [56:28]
B
Definitely something to keep an eye on.
- [56:30]
A
Yeah, yeah, absolutely. My side, I think. Again, not to plug here, but it has been interesting. The Stripe projects thing, people are asking, does that make them like a marketplace in a cli? I can't give the answer to if it's definitely yes or no on that one. But I think what we are seeing, and I think we're going to see more of this, are other players coming into this CLI driven market. I think it was sort of surprising that to some people that Stripe had decided to branch over here. Perception was quite positive. So I think we're going to see more of that. I think we're going to see more companies that never would have thought to go back to their CLI and maybe think that that's where they now need to target some product releases. For example, CLIs are kind of getting a moment again, especially because of the, you know, MCP versus cli. That's actually a whole other topic we should cover maybe next month. But CLI is coming back in a big way.
- [57:23]
B
Oh yeah, CLI is. It's crazy. Like everything I think is now about the cli and I wonder, you know, I think there's probably a bigger topic conversation around CLI versus MCV versus, you know, APIs and all this sort of stuff. But yeah, CLI is definitely having a moment right now, which is interesting.
- [57:40]
A
Yeah. So I'm sure we'll something to that effect will pop up, I think by our discussion next month. So thank you again, everyone tuning in to SED news. We hope you have a good April and we'll catch you next month.
- [57:54]
B
Yeah, thanks everyone for listening. We'll see you next month.