Loading summary
Transcript56 lines
- [00:00]
Mike Taylor
Hi, I'm Mike Taylor. I'm the head of tech consulting at every, and I sat down with Kyle Daigle, the CEO of GitHub, and talked to him about what is happening on the front lines of coding agents. We have 17 million pull requests coming in every month to GitHub now. It's growing exponentially, and that puts them at the forefront of what's happening in this new economy.
- [00:21]
Interviewer (possibly Dan Shipper)
We talked about how that affects our
- [00:23]
Mike Taylor
users as well as how this affects open source maintainers. And we covered a topic which is dear to everyone's hearts. How do I stop my $200 a month coding agent subscription from ballooning into a $2,000 a month usage limit? In this interview, we did something a little bit different, which is I told Kyle I had made an AI clone of him to practice the interview, and he revealed something surprising in return. Here's the con.
- [01:00]
EVERY Subscription Promoter / Podcast Host
EVERY is the only subscription you need to stay at the edge of AI if you care about being on top of the latest models and using the latest tools. You have to subscribe to EVERY to separate out the signal from the noise. Go to EVERY to subscribe today.
- [01:12]
Interviewer (possibly Dan Shipper)
Hey, Kyle, thanks for spending some time with me at the conference.
- [01:15]
Kyle Daigle
Yeah, of course.
- [01:15]
Interviewer (possibly Dan Shipper)
Yeah, it was good to meet you as well, and the day before. And I feel like we already kind of covered a few of these questions, but I think it would be good for the wider audience so they can understand what's going on here. Absolutely, yeah. So first thing I think is we were talking about is really interesting is that the demographics of the customer are changing. A lot of people who previously Maybe never used GitHub or never used developer products before are now using them. So how has that changed the way that you decide the product roadmap?
- [01:46]
Kyle Daigle
Yeah, yeah. I mean, I think for GitHub in particular, we've always really had this really expansive view of what a developer is. I started as a developer before I would have ever called myself a dev, you know, where I was just like, writing code. But it was just for me. And I went personally, like, a completely different career path. I didn't go to school for computer science. I was going to art school. I wrote code to pay for art school, which was a very silly decision as an adult now, I guess. But then that sort of journey of just, like, I can create tools with the team and deliver them to people who can have that same experience of like, I just want to build an app that's for me or for my family. Maybe as a startup, maybe as a business, we very Much have serious developer tools. All the largest businesses are using GitHub, but when I look at something like the GitHub Copilot app, I see just as many developers that are using AI every day, running multiple projects, all kinds of agent sessions at the same time. And I see our legal team at GitHub using the GitHub Copilot app or the finance team or I was meeting with a customer today and they were saying the same thing. A lot of the folks that the industry would call knowledge workers or just non by trade developers are using these tools to build little apps or assets for them. And so while our focus is very much on developers, I think we want to make it easier for people to choose to try to write some code and make sure there's always an on ramp into, you know, writing some software now with, you know, things like the GitHub copilot app.
- [03:26]
Interviewer (possibly Dan Shipper)
Yeah. And then how do you deal with the. Or how do you help developers deal with the burden of all of that extra. You know, there's a flood of PRs now like open source maintenance I talked to are like drowning.
- [03:35]
EVERY Subscription Promoter / Podcast Host
Yeah.
- [03:36]
Interviewer (possibly Dan Shipper)
How do you, how do you like what needs to happen to help them?
- [03:38]
Kyle Daigle
Yeah, I mean I think for all developers we're, you know, building tools like the Copilot code review. It's now Agentix so it finds a lot more novel vulnerabilities and you can just like comment and the agent will take that on and go implement the change if you want to. So I think that code review step is in some ways overlooked as a really great way to get PRs to a place that are much more easily reviewed. I think that the agentic merge in the app is another place where we see a lot of times internally and in the community, you may comment on something that might have a code review and you might go through and get it almost all the way there. But then there's all those like manual steps just to finish processing the PR instead. I can go in and set exactly what I want to allow, you know, GitHub copilot to do and say, okay, now go merge this PR and wait for CI and wait for policies and all of that. I think that's a big part on the open source side. It's a unique set of needs because you don't control who's, you know, sending everything in or you haven't really historically, and that's been really where we've been focusing is giving maintainers more tools to decide what. Well, do you want to accept all of these PRs like, who do you want to accept them from? How much work do you need to do to kind of prove that you're going to contribute something that is going to be meaningful to this project? And that's something that we want to provide tools to open source maintainers, but really leave them in control. Every community is choosing a slightly different way to approach the problem. And for GitHub, we've always wanted to leave that in their hands, like give them tools and enable them. But if a standard comes out of that or most are using a certain practice, we'll lock that in. But we don't really ever want to be the first to create a standard or an approach. Like, I think, you know, Mitchell Hashimoto shared like the vouch system that they use and I was getting questions like, well, why aren't you roll this out to everybody? But there's just as many communities that don't want to use that system because they have their own ideas of how it should work. And so for now, we're focusing on the building blocks of controls for maintainers. And then as we all are kind of learning together and as maintainers send feedback in, we'll cement an entire system if one emerges.
- [05:56]
Interviewer (possibly Dan Shipper)
Yeah, and I feel like you have a front row seat to this new agent economy, where I think you said public on Twitter that you've had more pull requests submitted per month than you did all last year. How are those stats exploding?
- [06:13]
Kyle Daigle
Yeah, I mean, we're seeing way more activity on GitHub. You know, we've always been talking about our users, you know, for many, many years and that growth. But this year we're seeing obviously the growth of developers having agents building with them. And so last year In October at GitHub Universe, we shared, you know, there's a billion commits on GitHub for the full year. We're on track to be 14 billion if the growth is linear this year, which it will not be. In March, there were 17 million pull requests that were created by agents. That's just the agent pull request. And so there's so much more code being created. And I think at times everyone goes like, oh, this is all just slop. This is all just code that's being pushed up and no one cares. It's not really true. We're all just actually getting to the point where we're no longer in the super early adoption. We're definitely not at the peak, but we're climbing that hill, you know, to see what can we build when it's not just Kyle Building. But It's Kyle in 12 to N agents that are using my skills, using my resources, using my context, and so on and so forth. And so like we're investing heavily in preparing for the next wave of growth because it doesn't seem to be kind of like growing and plateauing. It's just going to continue to grow. Because no matter where you're building or what tools you're using to build, all of that code ends up on GitHub or that's where you're sharing it with the world. That's where you're collaborating in a pr. And so we need to be able to support everyone's sort of agent moment and not just GitHub. Copilot.
- [08:01]
Interviewer (possibly Dan Shipper)
Yeah. How does the business model change? Because I think freemium makes sense in a human centered world where we go to bed but the agents are still working while we're asleep now. So does that change to usage based? You see that kind of where things are going?
- [08:18]
Kyle Daigle
Yeah, I mean, I don't think we know yet. Ultimately, I think we very much right now Kyle's going to have a license or Kyle's using GitHub.com for free. And we've always had API rate limits and things like that. And that's usually where folks are seeing the agent back pressure. I think, I think the goal is that if you want to be able to do way more, if you want to be able to, like Peter Steinberger says, 150 agents are doing everything all at once. That's great. We want to be able to enable that to you. But at the same time, I want you to have a great core GitHub experience. And at the very least there's some amount of agent usage as part of that is necessary. Similar to how we way, way, way back.
- [09:08]
Podcast Host / Dan Shipper
Right.
- [09:08]
Kyle Daigle
You'd have free public repos, but you didn't have free private repos. And then we said, okay, well actually it's fair for an individual to have some code that they don't want to put out into the world and we'll give you free private repos to allow you to do that. So GitHub's always evolving as the industry and community does, but we're always sort of focused on I need to make sure you, the dev, have what you need to be successful and then work with enterprises to make sure they have what they need to do at scale, which is usually a little bit different than what an individual dev is doing.
- [09:43]
Interviewer (possibly Dan Shipper)
Yeah. And I guess the business model pricing kind of, that all leads back into the Wider Microsoft orbit because you have a dual role now. Right. Partial responsibility for the wider kind of marketing Org. So do you want to talk me through how that's changed and how you prioritize between those two?
- [10:00]
Kyle Daigle
Yeah, I mean, so I've been at GitHub for a very long time, 13 years. And you know, as a developer myself and leading engineering teams for a lot of that time, and I think what's always been unique about GitHub is we really, really focus on the dev. Like we're building tools for the developers and the fact that people like enterprises are buying them is awesome and that's definitely great. But we're not building for the buyers, we're building for the developers, 100%. And that's been my focus as the COO of GitHub, which I continue to do then. Now as the Chief Marketing Officer developer for Microsoft, my goal is to look across all of Microsoft's tooling, their developer tools, their technology that they're bringing to developers and making sure that we're bringing holistic solutions that you can use that are authentic to developer experiences. And at events like this where we've taken a very different approach to build, this year we're in San Francisco. First off, the vibe is a bit different than the conference hall setup. Really focused on can I go to a session, can I use the thing? I don't want to be pitched on a thing, I have to be able to use it, Expo hall, so on and so forth. It's really bringing that expertise and, you know, love and focus on the developer that GitHub's always had to have even broader impact throughout all of Microsoft.
- [11:31]
Interviewer (possibly Dan Shipper)
Yeah. And did I hear you say that this is the first build that you've had external contributors and it's the first
- [11:37]
Kyle Daigle
build that I think by intention we focused on having, you know, speakers, you know, from the community. Like in these primary sessions, you know, that includes, you know, in the keynote we had a bunch of folks like Peter, you know, there's sessions from, you know, SWIX and others as well. I think that it's important, like software development is a team sport. You know, it seems silly to think that there's any one company, one group inclusive of GitHub and Microsoft and everyone that can just answer every single question that's not how software gets made. You know, we're all at least using open source and we're building on the backs of these giant open source projects. Let's invite people in that can help tell their part of the story together because I deeply Believe that that's what developers want. I know that's what I want. I know that's what my friends that are developers want. And when we look at the events and we hear the feedback, they're excited to see people from Microsoft, from GitHub, and then, oh, I get to see this outside perspective at this event. It's really meaningful.
- [12:45]
Interviewer (possibly Dan Shipper)
Yeah, that makes sense. And it's a very competitive market, right? Sure, yeah. Like the most competitive market, probably maybe the last competitive market. I'm not sure. But how do you differentiate in all of that, given the pace of change is so quick?
- [13:00]
Kyle Daigle
Yeah, I mean, I think we continue to focus on our roots, which are we care a lot about developer choice. That's always been true. We care about building for builders and enabling builders. And so I think we're in a moment that's really interesting because we've went from an era of having a ton of APIs, all this access to a little bit of an unintentional walled garden setup where you get a kind of affinity. Sometimes I'll say it's a little bit of a mousetrap. And then you realize, oh, this thing's really interesting over here. And then I have to, oh, I have to go learn a new thing or a new tool or a new account. I think for us, we always want to enable developers that are building with GitHub to go use these other tools and we'll partner with everyone to make that as simple as is possible. While I think that there's other folks that are doing similar things, I think the ability to do that across the entirety of building software, and not just the Cogen side or not just the collaboration review side, but across everything, is a real, you know, superpower of ours. And so, you know, I think you'll see us invest in our own tech. Like we talked about the new Microsoft AI models, you know, that we'll continue to bring to developers, we're also continuing to partner with anthropic and OpenAI and Google, and kind of anyone who's bringing a model to market or a coding agent to market, we'll partner with you and we'll both let you bring that to us, or we'll admit it through GitHub and GitHub Copilot. That choice is core, and that's something that I don't think will ever back down on, because if we do, developers will still choose. They'll just be stuck in another kind of mousetrap. And we don't want the world of software to be like that.
- [14:55]
Interviewer (possibly Dan Shipper)
Yeah, yeah, yeah. And how do you like, when you're making the decisions internally? Like there was a news cycle recently about how cold code licenses are being canceled and like, how do you make the trade off between dogfooding your own products, like using the new models you made or using the GitHub Copilot app, desktop app, versus letting developers experiment with other tools?
- [15:19]
Kyle Daigle
Yeah, I mean, we all are using a variety of tools because otherwise you lose track or you're too interested in your own work. So for me, I've been a daily driver of a MacBook for many, many years. I use Windows PCs on the weekends when I play video games. And I got this role and I have my Mac, I have my PC, and I have my Omori Linux box. So I can make sure that every weekend I code. Most Saturdays I do my kids sports activities in the morning and then in the afternoon I'm coding and I'm swapping between the boxes because I want to understand like, okay, what's that experience? The GitHub Copilot app I only use on Windows because I want to make sure that developers who are on Windows also deserve great apps. It's not just the audience that's on a Mac, for example. And that's true across our teams, especially when we're looking at, okay, what about the coding agents, what about the harnesses, what about the desktop apps? What about memory management? What about everything? We have this really great culture of just experimentation. Everyone is building and using these tools. Well, obviously we're putting most of our energy into our own tools. And it's such a blind spot that I think it's happened to GitHub in the past where like, when you're doing something and you're doing it well, you really laser focus. And that's what every piece of startup energy says, right. It's like look down and just keep moving and keep moving fast. And I think that's myopic. You know, I think that while I can't spend every day using every tool, when something comes out, I want to know why this is really great. Why are people having a great experience with this? Not only so I can understand, but so I can figure out, okay, well, for our goals, for our goal of developer choice, I don't need this. Like, I want to focus over here, but I want to know why a dev would pick these tools. And the same thing goes, you know, the same thing goes for our teams.
- [17:19]
Interviewer (possibly Dan Shipper)
Yeah. And how do you, how do you filter? Because obviously a lot of these ideas are relatively short Lived enterprise product development cycles are longer lived. How do you decide?
- [17:28]
Kyle Daigle
Yeah, I think that like right now we're in a moment where we're really looking at the like the short term in capturing the ability to have a multitude of, you know, agent sessions. This idea of just, you know, because that seems quite clear, you know, everyone's doing it, how can we cement it? But it seems clear on the longer term path models are going to continue to get better. The prices of tokens, like token economics is going to be a bigger, bigger factor in like what models everyone is using. And I do strongly believe that were not very far off from having the serious ability to use something above a small language model on a local device to be able to do some of our work. And so if I assume that I have all this optionality when it comes to tokens effectively the thing that I think seems to be true from the beginning to open claw to now is this idea of personalization or mine or context or fine tuning with context or memory. All of these ide seem to be a truth that's been there since you know, ChatGPT came out or GitHub Copilot came out and there's experiments but not a long term like vision I think for this like across the industry. So I think it's a good example of like where I need to get you to use agents incredibly well. A lot of them because like if you're, if you're into using agents, you're not just going to be staring at a single agent working. But that's not going to give you a long term great experience. Using an agent that you feel like is completing a thought for you will give you that great experience. Especially if you did not have to personally codify that thought to your agent. Always remember that I insert things. That's a lot of work, 100% like it should be able to intuit that or potentially again like you know, post trainer, fine tune or frontier tune, like a model that deeply understands me and how I'm using the work. That is kind of how we're looking at it. Sometimes it's short term and sometimes we got to take a bunch of bites of the apple or a bunch of attempts at the long term to get to something really tangible to help us move forward.
- [19:44]
Interviewer (possibly Dan Shipper)
Cool. And I heard the term hill climbing like 100 times yesterday and I'm a big proponent of that because experimented with DSPY auto research, a few others. Can you talk a little bit about how that's become a big focus?
- [19:58]
Kyle Daigle
Yeah, I mean you know, I think, you know, Satya and Mustafa talk about it a fair bit and Jacob to leading the copilot group. The biggest thing that we've kind of learned is we need to use the, you know, the use of the tools as a core way to improve the underlying use of the models, our own models, etc. In just the evals that are necessary to ensure that we're actually improving from things like, you know, using the thumbs up, thumbs down data that comes in to using like whether you're accepting it and how much you're accepting, you know, all of that data is enormous to create that magical type of experience that's not just for you but for everyone. And so every week we're talking about the hill climbing results. You know, we're looking at the data, we're looking at the improvement, we're looking at both the hard measures and the soft measures because sometimes the hard measures and evals and rubrics will show that we've made an improvement. But like user sentiment will crash. You know, even with the same latency and performance, it's overfitting basically 100%. And so like being able to really do that loop incredibly quickly. And then I think the main goal is giving everyone one of these hill climbing machines and not have you have to do it the kind of hard way that we've all been doing it. But particularly if you're in an enterprise and you are using M365, we know so much about that data or we could know so much about that data because of all the assets, all the documents, the chats and so being able to turn on something like Frontier tuning and using Mai Thinking one as the base model, it shows real results without having to do all that extra work. And it's been interesting because when I first heard about this, I'll be honest, I was like, this is like a magic parlor trick, you know, that is not going to be real.
- [21:56]
Interviewer (possibly Dan Shipper)
It can't really work.
- [21:57]
EVERY Subscription Promoter / Podcast Host
Yeah.
- [21:57]
Kyle Daigle
You know, and I think the reality is that's sometimes where the alpha is, is like where it feels like this is too simple to work. You know, we all have all this data and what are we going to do with it and we have to do all this effort to make it work. But I think so much has come down the pipe to allow us to just use the data and improve, look at the workflow and improve and just keep doing the hill climbing. That's why I think we say it so much, is that it's not these moonshots or like A hill climb. It is just climb, climb, improve new eval, improve new data improve and just keep going to get to the point where, you know, we're able to launch these models, seven models for ourselves and then, you know, allow customers to use the same or similar, you know, tooling to do it.
- [22:41]
Interviewer (possibly Dan Shipper)
Is that the answer to stopping the $200 subscription becoming a $2,000 subscription?
- [22:47]
Kyle Daigle
I mean like, I think, you know, $200 subscription to $2,000 is really going to be not only, you know, making these models or you know, frontier tuning these models so they way they know you better. But I also think it's really, really going to be about how can we, particularly for developers, you know, help you automatically choose the models and potentially either have like a model in that step,
- [23:15]
Interviewer (possibly Dan Shipper)
like the model router in the wallet.
- [23:17]
Kyle Daigle
Exactly. Like the, you know, auto model router with task content. And Microsoft Foundry has a model router as well that can do this sort of at an API level. The more and more that we can help you tell us a bit of where your bars are. This is an incredibly hard problem and I'm willing to go all the way to the top or I just kind of want to sit here and let us help choose the models because there's a lot of times where a lot of the reasons why tokens are expensive is because we're all going and choosing our model of the day or a week or hour, you know, and those models are incredibly expensive. But my train of thought is slipping in and out of a hard problem to like a simple problem. Personally, I feel like I'll, you know, get an agent to do like an enormous amount of work and then there's always that last step that is like a smallish thing, you know, like, oh, I don't actually like change all the naming of this to this.
- [24:13]
Interviewer (possibly Dan Shipper)
Yeah, and that's like don't need to
- [24:15]
Kyle Daigle
find and replace, you know what I mean? But, but am I going to actually go and like, oh, I want to save tokens right now. So I'm going to go off of 4, 8 or 55 and to haiku or something. Probably not, but the tools could. And I think that will really help us, particularly in the enterprise. But even for individual developers and folks that are building automations and using their copilot SDK to power that, it'll help them too.
- [24:43]
Interviewer (possibly Dan Shipper)
I did something a little bit weird. I hope you don't find it creepy, but I made an AI version of you to practice this interview.
- [24:49]
Podcast Host / Dan Shipper
No way.
- [24:49]
Interviewer (possibly Dan Shipper)
Yeah, and it's actually been pretty spot on so far and hopefully you think the questions have been good.
- [24:53]
Kyle Daigle
They've been great. I want to see what AI Kyle
- [24:56]
Interviewer (possibly Dan Shipper)
said and yeah, it's just in the terminal I didn't have like, I didn't go the full whack and make a video thing, but sure, sure, sure. But I found it immensely useful. I just wanted to ask what are the weird things are you seeing people do internally or externally?
- [25:10]
Kyle Daigle
Oh man. So it's so funny that you say that cause I do a very similar thing where I have both via the app and then I have a claw that can't talk to work stuff. So I have separation of where I spend a lot of time having it read everything I write and say. Like this interview will get fed into it ultimately. And every day I get a, like a comms report. That's not like what Kyle said, but like Kyle, you keep saying this.
- [25:46]
Interviewer (possibly Dan Shipper)
Yeah, okay.
- [25:47]
Kyle Daigle
This isn't super clear like you know, based on how you speak. Because I find that I write and speak in like a very particular way that like I want to use a lot of metaphors and so it'll just give me examples of metaphors that are clear. Yeah, I find that the like self improvement loop as a human from these agents to be incredibly powerful. We used to talk about it way back with Hubot at GitHub, like chat ops and we used to say like humans are way more willing to take critical feedback from robots than other humans.
- [26:17]
Interviewer (possibly Dan Shipper)
Yeah, it's less threatening 100%.
- [26:19]
Kyle Daigle
And so when my open claw that I affectionately named Baxter, you know, tells me how terrible I did in something, like I feel way better going tell me why and then ensure that when I'm writing emails, when I'm writing a script or I'm reviewing details that you're giving me that feedback. So a lot of my agent loop is really about me and less about like the software side. I still have all those tools too, you know, but it's always looking backwards. It's going, okay, the last seven days I'm going to read all Kyle's emails, slack messages, you know, and then give me feedback and then look back at what the agent told me to do. Did Kyle do it and go back seven days? That loop is like super, super, super powerful. And I think honestly like the type of personal consumer experience that I want out of AI, you know, to be able to tune these tools that way.
- [27:10]
Interviewer (possibly Dan Shipper)
Yeah, we need to recursively self improve as well.
- [27:12]
Kyle Daigle
100%.
- [27:12]
Interviewer (possibly Dan Shipper)
Yeah. Thanks so much man.
- [27:13]
EVERY Subscription Promoter / Podcast Host
I appreciate it.
- [27:14]
Kyle Daigle
Thank you, thank you.
- [27:15]
Interviewer (possibly Dan Shipper)
Enjoyed that.
- [27:24]
Podcast Host / Dan Shipper
Oh, my gosh, folks, you absolutely, positively have to smash that, like, button and subscribe to AI and I. Why? Because this show is the epitome of awesomeness. It's like finding a treasure chest in your backyard, but instead of gold, it's filled with pure, unadulterated knowledge. Bombs About Chat GPT Every episode is a roller coaster of emotions, insights, and laughter that will leave you on the edge of your seat, craving for more. It's not just a show, it's a journey into the future with Dan Shipper as the captain of the spaceship. So do yourself a favor. Hit like smash, subscribe and strap in for the ride of your life. And now, without any further ado, let me just say, Dan, I'm absolutely, hopelessly in love with you.