Vibe Coding at Heroku with Vish Abrams

Summary

Podcast Summary: Software Engineering Daily – "Vibe Coding at Heroku with Vish Abrams"

Episode Details:

• Title: Vibe Coding at Heroku with Vish Abrams
• Host: Software Engineering Daily
• Guest: Vish Abrams, Chief Architect at Heroku
• Release Date: April 24, 2025

---

1. Introduction to Vibe Coding

Vish Abrams introduces the concept of Vibe Coding, a term coined by Andrej Karpathy, which represents a new paradigm in software development where AI tools assist developers by generating significant portions of code. Instead of manually writing every line, developers act as directors, guiding and refining AI-generated solutions. This collaborative interaction blends human creativity with machine efficiency to tackle complex problems.

[02:52] Vish Abrams: "...developers act as directors, guiding and refining AI generated solutions rather than manually writing every line of code."

Kevin Ball echoes this sentiment, highlighting the shift from traditional coding methods to a more fluid and controlled approach using AI tools.

[04:04] Kevin Ball: "It's a little uncomfortable, but also kind of neat to see."

---

2. Current State and Limits of Vibe Coding

The discussion delves into the current capabilities and limitations of Vibe Coding. While AI tools are rapidly advancing, there remains a complexity cutoff where these tools may struggle, especially with newer or less common libraries and languages.

[05:12] Vish Abrams: "There is sort of an asymptote that we're going to hit that kind of like we did with self driving... the last 10% is super hard."

Kevin Ball points out that similar to no-code tools, Vibe Coding excels in prototyping but may face challenges when moving towards more complex applications.

---

3. Vibe Coding in Production Environments

Transitioning from prototyping to maintaining and scaling applications in production poses significant challenges. Vish Abrams emphasizes the need for new protocols and tools to bridge this gap.

[08:47] Vish Abrams: "How do you bridge that gap from day one to day two... giving your assistant the ability to start reading the logs out of your production application..."

Kevin Ball adds that integrating validation mechanisms like tests and type checking is crucial for ensuring the reliability of AI-generated code in production.

[15:15] Kevin Ball: "...add validation... formal ways to validate, you're looking for formal ways to validate."

---

4. Tooling and Protocols: Model Context Protocol

A significant portion of the conversation focuses on the Model Context Protocol (MCP), exploring its capabilities and limitations in both local and remote contexts. Vish Abrams discusses Heroku's efforts to expose their platform as an MCP service, aiming to give LLMs (Large Language Models) access to deployment concerns.

[25:23] Vish Abrams: "We're working right now on exposing the Heroku platform as a model context protocol service..."

The integration of OAuth for authentication and the need for standardized response formats are highlighted as critical factors for the successful implementation of MCP in distributed environments.

[27:43] Vish Abrams: "There was just an extension, a new protocol that focuses a little bit more on stateless and allows you to upgrade into a stateful connection..."

---

5. Team and Industry Implications

The advent of Vibe Coding prompts questions about team structures and the future of software development jobs. Vish Abrams suggests that while individual productivity may surge, collaboration between larger teams could become more complex.

[36:13] Vish Abrams: "How do you collaborate and deliver across groups in a situation like that?"

Kevin Ball introduces an economic perspective, positing that reduced costs in software development could expand the market, leading to increased demand rather than reducing the number of developers.

[34:07] Kevin Ball: "The cost of writing software is going down... So either we need to support fewer people or there's a lot more demand for software."

---

6. Skills Needed in a Vibe Coding World

In a landscape dominated by AI-assisted coding, certain skill sets become paramount:

• Problem-Solving: Ability to debug and isolate issues efficiently.

  [39:25] Vish Abrams: "The problem solving skill set is still strongly there... being able to quickly isolate where it's going wrong."

• Clear Communication: Crafting precise prompts and instructions to guide AI tools effectively.

  [39:25] Vish Abrams: "Being able to clearly communicate and specify what you need seems to also be very, very important."

Kevin Ball concurs, emphasizing the importance of ruling out assumptions and narrowing down problems methodically.

[43:22] Kevin Ball: "They spend a lot of time ruling things out... How do I narrow in or bisect towards where the problem likely lies?"

---

7. Future Directions and Challenges

The conversation concludes with a look towards the future of Vibe Coding and the evolution of software development practices. Vish Abrams is optimistic about the continuous advancements in AI tools but acknowledges the challenges in standardizing protocols and enhancing team collaboration.

[49:58] Vish Abrams: "We're exploring... seeing what we learn, see what areas we need to improve our API."

Kevin Ball underscores the potential for innovation but also the necessity for robust validation and human oversight to ensure quality and reliability.

[45:04] Kevin Ball: "How do we go about building a prototype?... We have a review cycle... a human in the loop to keep it steering."

---

Notable Quotes

• Vish Abrams on Vibe Coding's essence:

  [02:52] "Developers act as directors, guiding and refining AI generated solutions rather than manually writing every line of code."

• Kevin Ball on the comfort and novelty of Vibe Coding:

  [04:04] "It's a little uncomfortable, but also kind of neat to see."

• Vish Abrams on the evolving complexity of AI tools:

  [05:12] "There is sort of an asymptote that we're going to hit... the last 10% is super hard."

• Vish Abrams on expanding the software market:

  [35:36] "Having a set of infrastructure in order to make that kind of experience a real thing where you can build a tool that's a personal app..."

• Vish Abrams on team collaboration challenges:

  [36:13] "How do you collaborate and deliver across groups in a situation like that?"

---

Conclusion

The episode provides a comprehensive exploration of Vibe Coding, its potential to revolutionize software development, and the challenges that lie ahead. Vish Abrams and Kevin Ball engage in a thought-provoking dialogue, emphasizing the need for robust tooling, standardized protocols, and adaptive skill sets to harness the full potential of AI-assisted coding. As the industry navigates this transformation, the collaboration between human ingenuity and machine efficiency promises to redefine the landscape of software engineering.

---

Follow Vish Abrams on Twitter or LinkedIn:

• [Insert relevant social media links if available]

Connect with Kevin Ball:

• Visit his website at Kball LLC

Transcript

[Narrator]

AI tools are transforming how developers write code, and although it's difficult to pinpoint how much code is now AI generated code, estimates suggest it's between 20% and 40%, and this figure is poised to grow in the coming years. This evolution has given rise to a new coding paradigm in which developers act as directors, guiding and refining AI generated solutions rather than manually writing every line of code. This approach was recently termed Vibe Coding by Andrej Karpathy and it shifts the programmer's role from detailed coding to overseeing and enhancing AI produced code. It emphasizes collaborative interaction with AI, blending human creativity with machine efficiency to solve complex problems. Vish Abrams is the Chief Architect at Heroku and previously worked at Oracle and NASA, among other organizations. In this episode, Vish joins the show with Kevin Ball for a wide ranging conversation about the state of AI based tools, whether there are limits to Vibe coding, AI tools for individuals versus AI tools for teams, the Model Context Protocol, Heroku's managed inference service, and much more. Kevin Ball, or K. Ball, is the Vice President of Engineering at Mento and an independent coach for engineers and engineering leaders. He co founded and served as CTO for two companies, founded the San Diego JavaScript Meetup, and organizes the AI in Action discussion group through Latent Space. Check out the show notes to follow K. Ball on Twitter or LinkedIn or visit his website Kball LLC.

[Kevin Ball]

Vish, welcome back to the show.

[Vish Abrams]

Thank you.

[Kevin Ball]

I say welcome back because you were on not long ago, as I understand talking about heroku and the 12 factor application. But let's have you reintroduce yourself really quickly so folks who missed that episode know who you are and then let's get rolling.

[Vish Abrams]

Awesome. Yeah, so my name is Vish Abrams. I'm the chief architect of Heroku, been with Heroku and Salesforce. Heroku is a subsidiary of Salesforce for about six years now. Before that I built cloud at Oracle and I was very involved in the OpenStack project which we initially created at NASA. Some people may be familiar with that one. It was a big open source project back in the day before Kubernetes and the CNCF took over.

[Kevin Ball]

Awesome. So you wanted to come back on the show to talk about something that in my mind is very different from what you were last on. Right. You were talking about this 12 factor application which is like how do you structure and take this principled approach to app development? And then we wanted to then have a conversation now about Vibe coding, which is this hot new thing. It's very, very trendy right now. But maybe let's start when you think the word vibe coding, what does that mean to you?

[Vish Abrams]

So I think Karpathy was the one who originally coined the term, and it's taken on sort of a life of its own. I saw a post from someone where they were talking about how when there's a need for a new term to arise, the first term that sort of fits that mold is just gets adopted by everyone. And it feels like that kind of happened with vibe coding. And I think when he was talking about it, he was talking about being in sort of a flow state as you code and how that AI tools for coding can get you into that state where you're not really deep in the code thinking about all the details, but you're sort of looking down on what's happening from above and just kind of directing the flow of the code in a new direction. Which I think is a cool way to think about it, but it seems to be more associated with the concept of can you build an application, at least the first few iterations of an application, without ever having to write any code yourself? Sort of like you have a team of engineers, maybe they're in another country, and you're giving them instructions, and then they come back the next day with an answer. So there's a little bit of distance between you and the code, and you're seeing where you get to. Is that your take as well?

[Kevin Ball]

Yeah, I mean, I think there is this, like, sense of riding the wave and just kind of letting things flow and letting go of control, which, I mean, to me, as a longtime software engineer, is a little uncomfortable, but also kind of neat to see.

[Vish Abrams]

Yeah, the control factor is definitely, I think, where people sort of get on different sides of the issue. Right. I've seen some reactions that say, oh, this is just a fad. It's never going to happen this way. Once you start doing anything real, it doesn't go anywhere. And other people saying, no, this is the future. All coding is going to be this way. So I guess we're on this journey together to find out exactly how deep the rabbit hole goes, maybe.

[Kevin Ball]

Yeah. I mean, I think there's something really interesting to explore there, which is for a long time, there's been this trend in the industry towards no code tools and things, and they all kind of, over time, have gotten better and allow you to do more and more without having to think about the code. And at some point, you tend to hit some sort of complexity cutoff where you're like, okay, I want to do this thing now, and My no code tool does not give me an abstraction to handle that. Now I've got to bring a developer into it. Do you feel like Vibe coding has that sort of complexity cut off?

[Vish Abrams]

I think it does. And I am surprised consistently by how that complexity cutoff is being pushed forward almost on a daily basis. I mean, when I look at all the different areas where generative AI is coming, coming into the picture in video production, image production, coding, it seems to be moving faster towards something real than at any point I ever expect. I'm always like, ah, it'll be a couple years before you can do anything real. And then in a month there's another step forward. Right. So I hesitate to make any kind of firm opinion about how far along that complexity curve we will eventually get. But my sense of it is that there is sort of an asymptote that we're going to hit that kind of like we did with self driving, where it seemed like it was always right around the corner, right around the corner. But that last 10% is super hard. Right. And I feel like we're going to hit one of those. But I keep being surprised that it hasn't happened yet. So who knows?

[Kevin Ball]

Yeah, well. And I think there's something interesting here that feels a little bit different than the no code example or something like that, which is like you can use some of the same techniques that go into Vibe coding to gain productivity even in a more structured environment. So I think one of the things that we might look at here is first off, obvious use case for these things is prototyping. There's a lot of interesting stuff we can do there and maybe you can go all Vibe. I saw a guy at a discussion group coding a Doom clone and he was literally like, I want a Doom clone. Go. Here's the screenshot. Feed it back into cursor. It's not working. Fix it. Right. And it was like at that level of abstraction. I won't say that he got to a great Doom clone, but it was surprising to me how far he got.

[Vish Abrams]

Right. One of the things that's been super fascinating to me is that the Vibe coding skill set, meaning there is some way of thinking and understanding how systems work that I think makes Vibe coding easier for someone who's actually coded before. And not to say that people who haven't coded can't do anything. My wife actually recently was doing some real estate research and managed to get Google Maps to have pins of some houses she was thinking about investing in. And she got help from ChatGPT to go through the whole thing, something she never could have done on her own. She's just not technical enough to build that without the help. So it definitely enables some people who have no coding ability to get really, really far. But I'm impressed with the people that do have coding ability and are able to sort of prompt and ask the right questions and think through things differently in their plan, seem to get a heck of a lot farther with it. So there is definitely a skill set that you still need in order to be successful in this new world. And I think with the right technical skill set, we are going to take it pretty far. I think it will go beyond prototyping. As it stands right now, there is definitely a wall. When I think about most business software and the complexity involved and think about maintaining it over time, qa, those kinds of things, it's where you really start hitting the wall on some of this Vibe coding esque style of programming.

[Kevin Ball]

Well, and I think that's where we get into some really interesting territory. Right, which is how do you transition from vive coding to a production environment? Are there productivity boosts that you can get from this? You call it vive coding style. I remember, I think Steve Yeage at sourcegraph coined the term like chop, like chat oriented programming. This new way of interacting with a code base, combined with some set of tooling and engineering principles can still work in a production environment.

[Vish Abrams]

Yes, I think it can. And this is an area that I've been thinking a lot about, because we're thinking about Vibe coding right now as sort of the 0 to 1. Like can you get from nothing to a prototype or a concept very, very quickly? And it's obviously super valuable there, but how do we bridge that gap from day one to day two? And I mean, maybe there's a new term we need, maybe we need to get like vive operations or something. But how do you get that same productivity boost when you're thinking about maintaining an application over time, Dealing with security updates, productionizing it, scaling it, all of those bits. And I think some of the recent advancements or protocols that have been proposed, like model Context Protocol, to sort of give agents a little bit more accessibility to operational tools, is the right direction to push there. But I don't know that we've explored that space enough to see how effective they can be. If you give your assistant the ability to start reading the logs out of your production application and reading telemetry, does it start helping you debug production issues? That's an interesting space that I Don't feel like has been explored yet.

[Kevin Ball]

I think there is some hints in that direction that I think are really interesting. So I'm using cursor, which is one of the tools that is really big in this space for product development. And I will tell it, hey, write specs for this, write tests now, run those and look at the output and it will use those to debug at least in development, what's going on. Or like if you have your linter set up well, it'll run a thing in the agent mode and it'll be like, oh, this isn't meeting the lints. What do I need to do? I need to change it this way and this way and this way. Like that feedback loop can put it more on the rails. But yeah, how far does that go? How far can you get there?

[Vish Abrams]

I think there's some really interesting space around examining the limitations. I've read some really nice write ups of the places where this type of coding starts to break down. Like when is the LLM failing? Or when is it bad? And analyzing where that happens. The first thing that I discovered was that if I'm writing code in Python or Go, and it's fairly basic, using the standard library, things that have been around for a long time, it tends to be pretty darn good. If I ask it to write some rust using a new library that came out in the past year, it just starts making things up, right? So there are definite areas where you're on the beaten path and it works really well and there are areas where it starts to break down. One that I haven't seen personally that I read about is this, since you speak of unit tests, cases where the LLM will actually start rewriting the code to match the test instead of fixing the test or vice versa, it analyzes, it finds the problem in the wrong place, which is it feels very similar to a lot of new programmer problems. Like when you have a junior programmer, the same types of mistakes you see there, you often see in these tools.

[Kevin Ball]

And to your point, this pushes towards having a technical background being really helpful. So like a technique I've used there is, I'll be like, all right, I'm getting this error. Analyze the code and the tests and tell me where you think the bug is. Is it in the tests or is it in the code? And it'll go. And then I as a senior engineer can look at it and be like, that doesn't seem so good. Go off in this direction. You need to kind of have a human in the loop. To keep it steering.

[Vish Abrams]

Yeah, I think that's where I've come to as well, is there is this value. But it does mean that your skill set is a little different from how it would be if you're writing the code right. You now have to think of. There was this awesome interview with Jensen Huang on Huge if True, and he talked about how he sees humans changing the way that they do things when agents are in the picture. And I think this applies to coding specifically. It suddenly you don't just need a deep technical skill set. You actually need more of a management skill set as well, because you're sort of directing an entity, not a person, but an agent, to do work. And the way that you successfully direct an agent to do work is similar to how you would successfully direct a human to do work.

[Kevin Ball]

So 100%. 100%. I have an example I can bring. Last Friday we had an interesting thing we wanted to try read a research paper. Interesting new technique. So how do we go about building a prototype? Well, I had a conversation with an agent and I said, write me a spec for this based on this paper. And it wrote a spec. The spec was so. So it wasn't great. And I was like, okay, you need to look at this document, this document, and cut these things out. And I'm doing this feedback loop as if I'm an architect or a manager. Once it gets to a place that I like, maybe I do some hand editing. I say, okay, now go and build it. And then once again, there's a review cycle when it does that. So it very much is that sort of tech lead or manager mindset to get it to implement, but on a much faster pace. We went from finished reading the paper to testing it in dev in 20 minutes.

[Vish Abrams]

That's the area that I found incredible productivity boosts in my own work, where even to the point of I just need to understand something. And normally I would have to find some examples, go through a bunch of documents, documentation. Like I'm building a prototype of how to deploy something in Kubernetes, right. That I have in some edge case of something that I haven't used before. And whereas before that cycle of learning and trying and prototyping would take probably a day, I can have deep research, for example, go out and collect a bunch of information and give me a first pass at it. That's 90% of the way there in five minutes, which is just a huge difference. And I do see people talking about how they tried out AI or agents, and they don't seem to be getting a lot out of it. And there does seem to be a skill set that you need to develop. It's like you need to know how to ask the right question and bring in the right information in order to get value out of it. And if you don't ask the question the right way or bring in the right information, then it just seems like a waste. Right?

[Kevin Ball]

Yeah, I think you're totally right. And there's a mindset aspect too, right. It's do I expect this thing to be a magic solve it for me box, or do I expect a back and forth and a steering and an experimenting and like kind of a probabilistic molding of this thing?

[Vish Abrams]

Yeah. One other comparison that I've made is it feels like when I'm working with an agent, at this point, it feels like pair programming, where I don't have my hands on the keyboard, right. It's like the other person is doing the typing and I'm saying, no, no, maybe you should change that thing. But I'm still, and I'm curious if you feel like, is that Vibe coding or is Vibe coding this other world where you're even more hands off and you're just not really looking at the code that's produced, or is there a spectrum?

[Kevin Ball]

Well, I think this is the key question, right? And to me, this is actually how you go from Vibe coding to production is you add validation. Instead of turning your brain off, you are paying attention, you're finding ways to validate, you're looking for formal ways to validate, which might be tests or type checking, and you're guiding the process. I think pure Vibe coding is like, whoa, I don't care about any of that. Just make this thing happen. Which I think as you highlight, it'll get you somewhere, but that's going to plateau very quickly.

[Vish Abrams]

I mean, I think I'm on the same page with that one. That's my sense. But I'm following with sort of bated breath just to see what the next new big innovation change. It's an exciting time to be in the industry because there is so much going on. I remember six months ago, a year ago, the whole story was around, are we going to replace programmers? We're not going to need programmers in the next year or two, if we're going to have AGI in next year. All of these things and. And my position early on, and actually Heroku's position as well, was that these kind of tooling was going to actually increase the number of developers because it makes it more accessible to more people. So it may be that there are less people with software developer in their title, but there are going to be a lot more people doing software development using these tools than there ever have been before. Which means you need a lot more people to operate or a lot more places to operate those things. Operation of software that more and more people are writing. I mean, if you have hundreds of people building personal apps that they just because it's so easy, they just built it for themselves, it solves a good problem for them. They're going to need a place to run that app at some point. That's worked for us so far.

[Kevin Ball]

Let's maybe dive down. There's kind of two interesting things to explore there. One is you sort of touched on industry implications and I think there's a lot of people feeling a lot of uncertainty about what does this mean for the job. So we can kind of go down like, what does the software industry look like as this plays out?

[Vish Abrams]

Yeah.

[Kevin Ball]

But the other that I think maybe we should touch on even first is like, what sort of tooling needs to be created to enable this boom in individuals vibe coding themselves personal apps.

[Vish Abrams]

Yeah. I think it's interesting and I actually know some people in this space that are very heavily trying to solve that particular problem. It's like, because you need a different set of infrastructure in order to make that kind of experience a real thing where you can build a tool that's a personal app, how do you give it your credit card to buy something? Like, there's a lot of trust issues when you start thinking about that level of personal assistance and apps that you could build. But I think one thing that's interesting, and you mentioned at the beginning of us talking that this seems very different from what we were talking about with 12 Factor. Right. But one of the things that I think has become more obvious to me over time is that at the base level, you still have an application running. Even if a person isn't writing code, that code is eventually getting written and then having to execute. So the underlying infrastructure, what makes a good application, what makes a manageable application, all of those things are really the same. They're just maybe a little bit more under the surface. And I think this is true of what we're seeing in the protocol and API development space too. The thing that makes agents able to use existing functionality, billing plugins, is having a good API, having it well documented. If it's easy to use for a human, it's going to be easy to use for an agent. Right. So you have to sort of solve all of the problems you had to solve for humans, for the agents as well. And I almost feel like there's going to be this big loop where everyone goes, oh, wow. So we actually have to continue doing what we were doing anyway, which is funny.

[Kevin Ball]

You are spot on with this. And one of the things it reminds me of is one of the things I've discovered as I use LLMs to code more and more and more is the same practices that are good software development practice for humans make the LLM do a much better job. Do you decompose your concerns well so that it's clear what does what and how things work? Well, the LLM is going to be able to generate code a lot easier if they're well, decomposed. Did you document it? Do you have tests? Like all these different pieces support LLM driven coding in a production environment, which is not really vibes only. It's got those layers of validation, but that's how you get to value.

[Vish Abrams]

I think there's an interesting tool set too in terms of improving. It's along the lines of what you're talking about, of setting up the right structure that an LLM needs to live within to build good software. Early on, what I noticed, and this is harkening back to the problem of I was trying to write Rust code and it was having a big challenge with it. I think in many cases in order for the LLM to be successful, you need a set of information to feed to the LLM. And you said to use cursor. It might be in your cursor rules file that helps it do a better job. And I think one of the things that we haven't really solved yet as an industry is how do we get that information into the LLMs in the most effective way? I don't know that it's just throwing a bunch of extra stuff into the prompt and saying make sure to dry and use these libraries and stuff like that. I think there might be something more to that. One of the things I've been pushing for is leaning on the open source framework communities to try and build some of these things I imagined. I called it a framework knowledge base. So EmberJS might have a set of information that could be used either in rag for the LLM, it could be used to fine tune LLMs. But as a community, if we can start building up these repositories of information of how to do good coding in a particular language and framework, then the LLMs are going to get Better. And then we can start now and maybe a year from now we'll figure out, oh, there's a new way to put memory into the lm and it'll happen a different way. But we need that knowledge base. It's like we need better data is kind of what I'm getting to 100%.

[Kevin Ball]

And I think one of the things that I've seen is like, you want to integrate a new library, you go to their docs. But docs written for humans, while good, are not actually optimal for our LLM context. And so we have actually a process at my company where we'll take a set of docs and then we will pull it down and we'll say, okay, write a tutorial based on this with a bunch of code examples for this, this and that, and then that becomes our context. That anytime we're referencing it, we reference.

[Vish Abrams]

Oh, that's fascinating. So I mean, I wrote an article about this, gosh, almost a year ago now, and it didn't ever really get enough attention for a couple people went, yeah, that sounds like a good idea. But things take time to gain moment. But I feel like there's got to be some best practices happening inside of companies around this thing. And could we do this as a community? Could we get open source behind sharing these things and having them out there so that everybody benefits?

[Kevin Ball]

Totally agree. Like, I feel like every docs page should have a link to the LLM docs, for example, right. Which are going to be heavier on code examples and a few other things that help the LLM do well.

[Vish Abrams]

Yeah, that seems like an obvious direction to go. It just. There's so much frothiness in the space right now. It's hard to get anybody to focus on the same things and collaborate well.

[Kevin Ball]

And I think another interesting area to go into, and I'm curious if this is something you're thinking about at Heroku. It feels like a lot of the tools right now are focused on the individual coder, right? So cursor or lovable or all these other different things is like one person is going to come and they're going to prompt this thing and create an output. But I feel like there's also a whole layer of team level challenges, right? Like how do you deal with sharing what are useful prompts, what are useful things? If the code is no longer the starting source of truth, there's a chat that is the starting source of truth. How does that get represented? How do we collaborate as teams in this?

[Vish Abrams]

That is a fascinating question. And to be Honest, I haven't thought about that one in detail. So ideas are bubbling up, as you say it, about what is the optimal way to collaborate with multiple people. I mean, the direction I was init initially thinking of is do you use shared deployments of applications as a way to start having that shared state? You know, you have the repository which is shared, you have the deployed application, maybe you have a staging environment that's shared and can you start feeding information in from those places? But it also I recognize there is an interesting question around the team collaboration and how you get the where do you store the artifacts that you use to produce what you produced? Because that can be useful. I think about all the times I go continue a conversation that I had last week with an LLM because there's a bunch of relevant context into the next question I want to ask that are kind of sitting there, right? And that gets lost if it's just a person sitting on their laptop doing it. Fascinating idea. I think one interesting place where this gap is becoming a bit obvious. I mentioned earlier model context protocol. It's an area that we've been looking at a lot because of all the interest. When Anthropic released it initially there was this big question in the industry is like, it seems sort of like this proprietary thing that they did and it could be useful to have a standard, but everybody was waiting for everyone else to kind of adopt it and use it and then suddenly everybody went oh, this is the standard. Now everybody's doing it and that we're seeing all these releases about it. But the thing that is most fascinating to me is how heavily the standard was focused on a use case of running on an individual's computer and using standard I O to expose the tools. The protocol was clearly written for that use case. And almost all of the open source tools that you see out there are focused just on the standard I O implementation. And coming from a distributed systems background and thinking about deployments and clouds, I'm like, this is very strange because I think about a tool as something that's remote and that is used like an API. And even the spec itself is a little light on that use. Like it didn't have authentication built in, it uses JSON rpc, which is a pretty flexible protocol. But it seems it was clearly built off of the idea of having a stateful request response on a single person's machine instead of a remote stateless protocol which would be more appropriate for a web service. And that's an area we've been digging into a little Bit is can we make this a little bit easier for people to use in a more remote context?

[Kevin Ball]

I would love to hear more about that because we had an example of that once again coming from my workspace where we prototyped something using mcp. We're like, this is great, we should integrate this into our agent which runs in a cloud based distributed environment. And it was like, oh, now there's a whole slew of things we have to handle that MCP doesn't give you out of the box.

[Vish Abrams]

Yeah, I think one of the interesting choices was to lean on the SSE HTTP protocol for the remote part of the spec, which has been around for a while and it supports sort of streaming and notifications, which I think is one of the things they were trying to achieve. But it's not very used in the industry. Like it's much more common in industry to do something a little bit more like what OpenAI was doing with their tool calling. And I think there was actually initially, a long time ago, the plugin model had a JSON openapi spec that you kind of put alongside your plugin in a well known location. And that feels a little bit more like what you would see in a remote request response because most APIs are already built that way. So it's very easy for someone to throw that additional endpoint in there with a little JSON schema saying what the API does and just do tool calling that way. This direction leans a lot more on the stateful side, but there have been some changes. I don't know if you've been keeping up with the changes going into model context protocol, but there was just an extension, a new protocol that focuses a little bit more on stateless and allows you to upgrade into a stateful connection if you want to, which I think gives some more room for those type of use cases. It still uses JSON RPC because that's sort of what the underlying specification uses. And there is also some work, I don't know if it's merged yet around supporting OAuth as an authentication mechanism, which I think is a huge thing. And now that we've added winsurf Cursor, all of the tooling has added model context protocol support. If OAuth gets in the spec, then your tool will be able to use a remote server using OAuth and you'll have a pretty simple flow where you'll just say there's a server over there, click a button, authenticate. It doesn't necessarily solve the service to service authentication issue, but it Definitely user to service authentication is much easier in that world. And I think we'll get to service to service and more standardized way of authenticating and connecting these things over the next few months. Just because so much is going into this right now and it will all be easier is my guess.

[Kevin Ball]

Well, and my associations with Heroku are back from like 2010ish or whatever. And it was like that was. The principle is suddenly this is easy, this is easy to do. So I'm kind of curious, are there other big gaps you're seeing people trying to deploy agentic things or all this stuff where it's not easy now, but you see a route to making it easy?

[Vish Abrams]

I think that's the general space that we're trying to be in. So we're working right now on exposing the Heroku platform as a model context protocol service so that we might be able to give LLMs access to deployment concerns instead of just development concerns, which I think is a really interesting space. And I'm sure we'll uncover cover some interesting issues and learnings from doing that. But then we're also focusing on just trying to make it super easy for people to deploy model context protocol servers on Heroku. And it does involve solving some of these remote stateful problems. I mean, at the very least, if model context protocol servers are always stateful, then they actually don't follow the 12 Factor App principles because the 12 Factor App is supposed to be stateless. So you're kind of painting yourself into a hard to maintain corner there. Then we're trying to make that easier by building some bridges and essentially where there's rough edges in the spec, just building in maybe some new ideas and some polyfill mechanisms to help push the spec forward. So we're trying to participate in the community and get involved there. And I'm excited about where it can go. It's got the attention and I'm super familiar with this kind of development because OpenStack followed this exact same model. When we released OpenStack, it didn't work. It was an idea and it got a ton of interest because there was a moment in the industry where everybody was saying, oh no, what do we do about aws, right? We gotta compete here. They're way ahead. How do we catch up? Let's all collaborate on an idea, right? And I feel like the MCP thing is in the same space where it's not quite there yet. The technology's in process, it's improving, but the idea is sound and everybody wants to collaborate on the idea and participate in the idea, which is a great recipe for really pushing technology forward quickly.

[Kevin Ball]

Well, and the fact that they started with something that more or less works in a local environment means that as an indie hacker, you can go wild because you only care about your local environment. It's when we start taking it more towards production use cases that all of these edge cases become important.

[Vish Abrams]

I totally agree with that. Having a nice local development story is super important for getting any kind of attention. One of the areas that Kubernetes struggled for a long time was how hard it was to use locally.

[Kevin Ball]

I have some battle scars from that.

[Vish Abrams]

Yeah, it is a good platform to build off of having that as a starting point. It's got the momentum. We just need to get some tweaks in. And I mean, as with any protocol, it's not necessarily about having the protocol perfect. It's about everybody using it. Right. It doesn't matter how good it is if nobody uses it. And that's the big lesson.

[Kevin Ball]

So we went down one branch of where we had been. Let's rewind back and talk about the other branch of what we had sort of touched on, which is like, what's going to happen to people who are software developers now? Right. Like, everybody's nervous around it. And you started to paint what to me sounds like a pretty optimistic picture.

[Vish Abrams]

I mean, nobody knows the future, Right. But I definitely have people asking me, like, should I stop studying software engineering? It just seems like there's not going to be an industry anymore. And I definitely don't think we're anywhere close to that. Right. But I do think the skill set's different. I mean, the skill set for writing is going to be different, the skill set for art is going to be different. Everything is kind of changing. But it doesn't mean that artists aren't going to have a purpose anymore, software engineers aren't going to have a purpose, or we're not going to have them. It just means, like, when photography came around, it suddenly wasn't all that impressive to be able to paint a realistic image. But. But art didn't go away. They just changed what they were focused on. And I think the same thing is true in our industry. It's like the value of being able to hammer out a bunch of code quickly may go down, but the value of building a system that works and is maintainable and operatable over time is still going to be there. Right. And the way that you build it just might start changing a little bit. That's my take.

[Kevin Ball]

Yeah. I Think that makes a lot of sense. I also like to do the econ geek thing, right? At a certain price point, there's some amount of demand. What we're saying is the cost of writing software is going down. So then either we need to have it's going to support fewer people or there's a lot more demand for software. Now I think software probably has infinite demand. Software is dreams made reality. I have an idea, and suddenly it can be true. I don't know any other place in the world where you can get kind of that level of creativity in terms of what's possible.

[Vish Abrams]

I think that's true. So I have a comparison that I have been making, which is so I consume a lot of content on YouTube. I'm so impressed by how you can learn things that just. You would have had to go find a mentor in some other country to go learn a skill set. And now you can just pick it up immediately by watching some YouTube videos. So I think it's fascinating. And what I've seen over the past, say 10 years of consuming that content is the, like the minimum bar on the quality of content has just continually been going up. But it doesn't mean there's less content creators. It just means they're producing much better stuff. Right? Because. And I think it's similar in what you're saying is there is sort of an endless amount of consuming of that kind of value, or at least we haven't saturated in it yet. And so if everyone is able to produce apps, right, the minimum quality of an app is going to go up, but there's still going to be a lot of apps out there. I mean, it just seems sort of obvious that that would be the case. It's not going to go away. You're going to have analysts and business people and marketing people building apps. You're going to have developers building apps. The floor is higher. They're all going to be better, right? Are there going to be less of them? Is there going to be less people doing it? I don't really buy that part of the story. It seems like there's going to be an equivalent amount or more. What do you think?

[Kevin Ball]

To me, I feel like there are so many, both personal and business use cases out there that don't have good software right now because it was too expensive to create it, right?

[Vish Abrams]

Ah, yeah, yeah.

[Kevin Ball]

Essentially every local store could benefit from a system customized to their inventory management or other things like that. And it may be it's not totally custom software. Maybe it's built on top of software, but it's some amount that's customized to their workflow. But that never happens because software engineers are expensive. And to build something that's solid, you gotta hire a team of five people for, you know, six weeks or whatever it is to build it and get it solid. But if suddenly that same level of quality as you highlight the quality of what an individual can produce goes up. So instead of six people for five weeks, you can hire one person for two weeks, and they work with an LLM to iterate and get you that piece of software. Maybe that's now in your budget.

[Vish Abrams]

Right. And so you're expanding the market significantly. Yeah, I think that's the key point I'm thinking about in my early days of programing when I was doing some work with nonprofits and I was like, building access databases and just throwing together the most janky interface for them to run their business or nonprofit off of. And how, you know, with that same amount of effort, I could build something much, much better for them that makes their jobs all much easier because now they can afford the, you know, the better experience. I think that's totally right on.

[Kevin Ball]

The question to me, though, is what happens to teams? Right. I feel like the industry for a long time has had this fundamental unit of a team size that I think is popularized by the Amazon 2 pizza team, right? And it's like maybe you have five or six developers, you've got a product manager, you've got a designer, a subject matter expert, something like that. This like, set of eight or nine people. And that's kind of a fundamental unit for a lot of at least product development. Do you need five or six developers if they're able to ride the vibe coding wave and produce 10 times as much in half the time? I don't know. If you do.

[Vish Abrams]

That is a very interesting question. I have an alternative challenge that rises to my mind from that question, and it's very personal because I've recently taken over managing the team of Heroku engineers, which is there's about 220 people on that team. And what I've noticed at a certain scale, all of the problems are the interfaces between the different groups as opposed to within the team themselves. Right. So you might be able to say, well, suddenly my 220 people only need two people on each team. But now you have 110 teams, right? And so how do you collaborate and deliver across groups in a situation like that? Because most large pieces of software end up needing that kind of interaction from people building different components and so the interesting question is, does solving our technology problems make our people problems worse? And are the LLMs going to be able to help us with that part of the problem?

[Kevin Ball]

Yeah, it's a real question. I could see a couple of different ways you could go, right? So maybe your 200 person team, which is made up of, I'm just going to round to maybe 20 teams of 10, maybe you still only have 20 fundamental units and those are now teams of two.

[Vish Abrams]

Yeah.

[Kevin Ball]

And so you have fewer people, fewer connections, it's not a problem. Or maybe as you highlight, you have a flowering, you still need 200 people or you have 200 people because that's great. But now instead of 20 teams, you've got 100 teams. That's a whole nother layer of challenge.

[Vish Abrams]

And it feels like in some ways it's kind of an economics problem. And maybe this leans back into the story that you were telling about how there's an unlimited demand. In this particular case, I've never been on a software team anywhere that ran out of things to do. Right. It's like you can always find if it suddenly costs 1 5th, go 10 to teams of 10 to teams of 2 in development cost to build something, you've just released 4/5, 80% new of new capacity, that if it's that much cheaper, you're going to find something for them to do to make the business money because suddenly the profit is higher, the margins are better. Right. So you're going to find more work and expand your but business. And I think what I've noticed is that economics problems are fundamentally super counterintuitive. Right. We just, as humans, we tend to think of everything as zero sum. And almost all actual economic situations really are not zero sum. And I think that's an interesting takeaway that I haven't thought about in depth. But the way that you put it about the unlimited demand, I think really is the key factor of that equation. Like there is a demand for software that kind of will just keep. They say it eats the world, right? It's eating the world because the demand doesn't stop. And so it does seem like the value and the productivity will increase. I mean, obviously there's going to be some shakeups and there's going to be some challenges. A skill set that was valuable 10 years ago May not be the same skill set that's going to be value 10 years in the future, but there is a whole new areas of industry that will open up because of that economic shift.

[Kevin Ball]

So talking about that skill set and Bringing us back to vibe coding. What skills do you think people need to be developing in a vibe coding driven world?

[Vish Abrams]

So the biggest one that seems to be relevant in my own experience working is the problem solving skill set is still strongly there. And I've seen a wide range of people who are good at debugging issues. You know, you have those developers on your team that are really good at sort of jumping into a production problem and getting right to what's going wrong super quickly. And that skill set is actually pretty rare. Not a lot of people are good at that particular part of debugging. And that part seems to be super relevant in this space because at least right now, the LLMs get into very weird corners a lot. Right. And being able to quickly isolate where it's going wrong and how to fix it, I think is going to be super important because it's amplifies your productivity. You can be stuck. It's kind of like that classic example of you spend all day debugging something and it turns out you missed a semicolon. Like you can. The smallest problem can cost you so much time if you're looking in the wrong place. Right. And I think the same thing is going to be true as we start using these agents as well. So one skill set I think there is the debugging problem solving skill set. Being able to sort of break problems into smaller pieces and really isolate what the source of the problem is is going to still be super useful. And then being able to clearly communicate and specify what you need seems to also be very, very important. When you ask the right question, you get the right answer. When you ask the wrong question, you get the wrong answer. I mean, it's a lot of the same skill sets you need when talking to a person. If you're telling the person the wrong thing to do, you're not going to get what you want. So clear communication, being able to understand and clearly specify what you need. LLM seem to do a lot better when like the English language is very ambiguous. Right. And being able to cut through that ambiguity and be very clear about what you need and how you need it to be done means the LLM can jump ahead much faster than if you just give it a very vague direction forward. So cutting through ambiguity seems another one. I'm sure there are more. Those are the ones that come to mind.

[Kevin Ball]

Have you seen a good way to teach people debugging? Because I too have seen this very wide discrepancy in debugging skills among people that may be very competent in other parts of engineering.

[Vish Abrams]

Yeah. So I did read a book that I felt super concisely explained the way that I think about debugging. And I have leaned on sort of that understanding. Sometimes you need to read something to know how to explain it to someone else. So I've leaned on some of the understanding I gained from that book when I'm helping people. But I can't say that I've solved the Rubik's Cube of conveying that skill. I think some people, their brains naturally work that way a little bit more. But the concise way I can find the book, I don't remember the name offhand, it's been a while since I read it, but it was talking about problem solving not just in a programming space, but in like. Like, if you're needing to fix your car, here's how you go about figuring out what to do, right? And the concise version involves, like seeing the whole problem space, finding somewhere in the middle, cut the problem space in half and determine which side of the line your problem is on, and then continuing to do that. Right? So it's like it's bisect. That was the gist of what it was talking about. And when I read it, I was like, this is what I do. This is why I can do this. I don't think I could have explained it to someone before I read the description, but it was super helpful in sort of understanding how I developed the skill.

[Kevin Ball]

Yeah, no, there is actually something there. I think one of the things that I've observed in folks who are better at debugging is they spend a lot of time ruling things out, even things that poorer debuggers assume. Essentially. Right. And so they'll do it. Like, what assumptions do I have? How do I validate them? How do I rule out this side of things? How do I narrow in or bisect towards where the problem likely lies?

[Vish Abrams]

I think that's the essential question is how do you throw out half the options? Right? And most people are looking for the thing that's going wrong. They're looking to find the needle instead of removing half of the hay. I guess if you just keep removing half of the hay, that's how you actually get there.

[Kevin Ball]

That makes a ton of sense. And also I suspect that's why LLMs are not that good at debugging, because they're very confirmation focused. They're constantly looking for a thing, predicting what is the next thing.

[Vish Abrams]

Fascinating. Yeah, that's good. I hadn't put my finger on that one yet. That's really insightful.

[Kevin Ball]

I hadn't thought of that until the way you described it. So yeah, that's interesting. Okay, so that's debugging. I think communication is another one where, I mean, of course we know all technical people are great at straightforward human communication.

[Vish Abrams]

Oh yeah. It's definitely a broad skill set in the software engineering community, but it is very learnable.

[Kevin Ball]

Right. Because it's a very common challenge. And there's both courses, there's groups if you want to learn to speak better, like Toastmasters and things like that, where they very much will work on that clear communication. I wonder if we will see a renaissance and like, hey, actually now our coders talk to the rest of the company.

[Vish Abrams]

That's interesting. That ties in with the team topic you were discussing earlier because, you know, it seems like it could go in a couple of different directions. One of them could be, well, you just can get so much more done with one person. So maybe you just give that super introverted engineer an LLM and say, go solve this whole problem by yourself. Or it could be that you end up with a lot more team bottlenecks between teams where you actually need to focus on that side a little bit more. That's fascinating.

[Kevin Ball]

I've been thinking about it in terms of constraints and sort of the law of constraints and how those things work. Right. Historically, a lot of software development was constrained on writing code that was the slowest piece. I think this moves the constraint around and very often now the constraint is on decision making. What is it we want to build, what is it we need to validate, what is it we need to test? And to me, that elevates one level of technical problem solving on the how do we want to approach this? But it also really elevates constraints in product and user research and design. Those become a much more limiting factor in how fast you can go.

[Vish Abrams]

But what about when agents are the ones doing the purchasing instead of the.

[Kevin Ball]

People that's moving a step or two down the road? How do their decisions work?

[Vish Abrams]

I think we're a little bit far from that, but I think you're right, there are new bottlenecks. I mean, anytime new technology comes in, it moves the challenge area around a little bit and we're having a really, really big movement here. And I don't know that we have even understood where the new places to optimize are going to be. As you say, if we suddenly have much more actual code output capacity, do the bottlenecks move into validation? Do they move into QA also? That's going to be. That's another area that could be a little bit harder to automate.

[Kevin Ball]

Yeah, it's a fascinating time. Well, we're getting close to the end of our time. Do you have anything that we haven't talked about yet that you would really like to bring forward?

[Vish Abrams]

I think one area I hinted on this a little bit, but I think one area that I'm really interested in, and it's something that we've been exploring a little bit, is what are the tool sets? What is the information that you need to provide to an agent to have it help with the areas around the main code production part of software? Right. Like, and I think we've hinted at it as in team collaboration and things like that. How do we get teams more involved? But I also think about it is like, I mean we've been through many phases, ops, DevOps, then Google, SRE. Like how you manage software over time has evolved and changed over the past couple of decades. But if you think about, if your goal is not just to make the code author better, but your goal is to make the SRA team better or the operations team better, or even if it's the developer who's operating the code, if you're making day two operations better, what actual information do you need to get to the LLM and what are the constraints and guidelines you need to put around that? I mean, I think everybody's worried about having an agent scale up and down their production deployment of something because suddenly there's a lot of dollars on the line that you don't want to necessarily offload to a human, but you need to automate some parts of that or else you're just increasing your operations costs. Right. And so where do we draw the line? Do we start feeding logs and metrics into the agent and let it make decisions? Is there a human in the loop there to approve or deny? Types of questions are I think areas that are going to become super relevant over the next few months as people start moving farther along in the deployment cycle with their agent developed code bases.

[Kevin Ball]

Yeah, that is a fascinating domain. So are you all tackling any of those particular pieces right now?

[Vish Abrams]

Yeah, so I mean right now we're just exploring. For example, we give an agent access to the Roku API. That means that the agent can say scale my dinos up and it could say let me check the logs to see if there's a problem. So there are capabilities that the agent suddenly has because it has access to the deployment system. We haven't done enough investigation yet. To figure out where the right guardrails need to be in that system. Right. We're in the more exploration. Let's give people the tools. Kind of like you're saying, you give people standard I. O they can play with it locally and figure out what's going on. So I think we're going to take the same approach. It's like let's just get the tools out there. Let's let people start trying out different things with their, like, you know, the seven or eight different coding agents that are out there and then see what we learn, see what areas we need to improve our API. I mean, we're thinking about is the API in many cases, as you're saying, the API that you give to a human might be the right thing. Agents work a lot like humans do. So well documented API, for example, is probably still valuable for an agent. But I think you made a really key point when you talked about the information about your documentation even isn't necessarily the right thing that you need to give to an agent. And I think the same thing is true in the operations world. You want to give a human the ability to tail their logs so they can look at something live. That's probably not the way you want to offer that thing to an agent because you probably don't want to stream logs in and have it be responding in real time to the logs. There's probably a different interface that the agent needs to see that information. Maybe you expose an API that's a rag endpoint for logs, just as a complete flyer of an idea. Right. There's going to be different APIs that you need to actually make the agent successful. And we're only going to discover those by getting the tooling into the hands of people to start playing with and then learning with our software development community.

[Kevin Ball]

Yeah, it's fascinating. And I mean, I think one of the things that I've seen both in terms of having agents write code and call functions is the more semantically distinct you can make your functions, the better the LLM does at it. So, right, like fetch logs with a bunch of parameters is going to do a lot worse than fetch last day's logs, fetch last week's logs, and having kind of exploding out the options into semantically distinct things that communicate in their names what it is that the agent would want to do with them.

[Vish Abrams]

Right. Because ultimately you're providing there's two parts. There's the actual tool and then there's the decision making, the reasoning that needs to happen about which tool to use.

[Kevin Ball]

And what parameters to pass it and all that other stuff. Yeah, yeah, yeah, yeah.

[Vish Abrams]

That's really cool. I'm also fascinated to see, like the MCP space is not just about tools. They have a few different other endpoints that they're exposing. And I feel like there's a little bit more definition in the same respect. So there's a difference between resource and a tool. And when you're exposing data, you could make a tool call that exposes the data, or you can make a resource that exposes the data. But I think the advantage of clearly separating some of these things and having well defined categories is the same problem of it makes it easier for the agent to make decisions. And I think we're just at the beginning of that. For example, I don't believe the MCP protocol standardizes the types of responses that comes like it standardizes the request but not the response. And so when you start thinking about chaining LLM calls together or agents using agents, that response format actually becomes kind of important. And if it's just free text, then it's a little hard to use the tool effectively. So I think we're going to start to see, maybe we'll have a standard web search schema for a tool call. And you might decide, I'm going to use Brave Web Search or I'm going to use Perplexity, or I'm going to use Google Web Search. But the way that the communication happens is in a standard way so that the tools can be optimized in their training set or in their fine tuning to make those calls in the same way. And I think that's going to make all of the tool calling and agent use better. But until we have sort of. It's so early, we haven't gotten to that point where we've all agreed on this is probably the best interface for this thing. We're kind of all exploring and trying different things. So I do imagine we'll start getting into a little bit more well defined interfaces around types of tools, calling conventions and things like that. That'll make the agents a lot more powerful in the future.

[Kevin Ball]

Absolutely. Well, and at the end of the day, we'll let you do more things just on vibes.

[Vish Abrams]

Vibe coding all the way. The vibes are good, good vibes over here for sure.