← The MAD Podcast with Matt Turck
Loading summary
Transcript48 lines
- [00:00]
A
I think we're underestimating how much you can do with the models that are out today, not the ones that are coming next year or next month. We are only touching the surface of what's possible.
- [00:11]
B
Welcome back to the MAD Podcast. Today we have a fantastic conversation with Thomas Domker, the CEO of GitHub. Now, of course, GitHub is the world's largest developer platform. Once upon a time A scrappy startup, GitHub was acquired by Microsoft in 2018 for $7.5 billion and today it serves 150 million developers and generates around 2 billion DOL dollars in annual recurring revenue. GitHub is also sitting today at the epicenter of the hottest space in generative AI AI coding, where hyperscalers like Microsoft and Google face off against AI giants like OpenAI and Anthropic and fast growing nimble startups like Cursor. In this episode we talk about the rise of AI coding agents, including GitHub's brand new copilot agent mode.
- [00:55]
A
The coding agent is like a new member of your team that can take on certain tasks and you can not only assign one task to one coding agent, can assign 10 tasks to 10 versions of that coding agent and they can all run in parallel.
- [01:08]
B
How to understand the current AI coding market and the delicate balance between competition and collaboration amongst players.
- [01:15]
A
It's the DNA of Microsoft to be competing and partnering with many companies in.
- [01:20]
B
The industry, as well as the future of software and code development.
- [01:24]
A
AI is going to accelerate the role of the professional software developer and it's also going to enable everyone that wants to become a developer to learn coding.
- [01:35]
B
This is a very educated, educational episode packed with insights for both developers and non developers interested in understanding where AI is going. Please enjoy this great conversation with Thomas Dumke. Thomas, welcome to the Matt Podcast.
- [01:49]
A
Matt, thank you so much for having me.
- [01:51]
B
Great to have you here today. I've been very much looking forward to the conversation because AI coding is such a fascinating area. It feels like ground zero for all things generative AI. First, it's arguably the most widely adopted application of generative A. It's where a lot of the innovation is happening. It's where some of the questions around is AI going to take our jobs or not? Feel the most real? And then it's like a big battleground as well from a business standpoint between the hyperscalers and the startups and the big AI labs. So what I'm hoping to do today is have a very educational and in depth conversation about all the things both for coders and non coders to understand where it's all going. Does that sound good?
- [02:37]
A
Sounds great, yeah. And it's also, I think, one of the oldest use cases for AI.
- [02:40]
B
Yes, perfect segue. Let's go back to in history a little bit. Let's talk about 2018, which was the $7.5 billion acquisition of GitHub, which had been a startup for over a decade by then by Microsoft. And you were deeply involved in this. Walk us maybe through the strategic thinking behind the acquisition at the time.
- [03:02]
A
7.5 billion seems cheap now that we're in 2025 and you see all these startup valuations. I think Microsoft has always seen itself as a developer tools company. It's a platform company. And to be successful with a platform, you need to have developers on your platform that build apps, services, products on top of it, but also build an ecosystem around the platforms. Microsoft, back 50 years ago, started with a BASIC interpreter for the Altair. Then you fast forward and in the 1990s, Visual Studio came in the 2000s, there was TypeScript and. Net and that ecosystem in 2018, a lot of things had changed under Satya Nadella's, Microsoft's current CEO, which came into hold in 2014, had changed that. Microsoft started embracing open source. Net was already an open source project. VS Code, the editor was already on GitHub and in fact one of the biggest projects on GitHub in terms of number of contributions. And so we felt it was time for Microsoft to fully embrace its roots as a developer company and bring in all these web developers, those that are building apps is now called cloud native, and bring them into the Microsoft ecosystem. So I think that was the primary motivation. Bring those developers that are on GitHub that are building in the open or on startup projects, their private projects and increasingly in the enterprise. As GitHub in the late 2010s more and more also embraced the enterprise and started selling a server, GitHub Enterprise Server and a business product in the cloud. And so bring that into the Microsoft ecosystem. And two years before the GitHub deal, the LinkedIn deal had happened. So 2016, Microsoft had acquired LinkedIn and had kept LinkedIn somewhat independent from the big mothership. And so we also had a template of how to do such a deal where we retain a lot of the value of the deal by keeping the company independent. And the three principles that we set back in 2018, they're actually still true today, which is GitHub always puts the developer first. And so that's what drives all the decisions the design, the product processes, how we think, marketing blogs, our conference we have Microsoft accelerate GitHub and I think that is a crucial part of how everybody should think about acquisitions, which is the buyer should accelerate the target company. It's almost like a venture capitalist investing into a startup, right? You invest money to accelerate the company you're investing into. When you're acquiring companies, I think you should have that same mindset, buying a company to accelerate them so they increase their value and as such I'm increasing the value of the buy side. And then lastly we also thought about GitHub accelerating Microsoft and here we are in 2025 and the the easiest example I can give you is how GitHub accelerate. Microsoft is of course GitHub copilot.
- [06:22]
B
So just to build on those two points which are super interesting, the GitHub helping Microsoft to which extent does GitHub fit into the Azure broader picture, the hyperscaler business model is generally described as building a bunch of applications that drive demand for the underlying compute. Is that one of the key objectives? What is the intersection between in GitHub in general, GitHub Copilot in particular, and underlying Azure business?
- [06:54]
A
The Azure business within Microsoft came out of division called Server and Tools. Today that division is called Cloud and AI. And the server and tools team also had the developer division. But this core product back in 2018 being Visual Studio in itself the IDE, the old school IDE in itself a billion dollar business and had a bunch of new stuff like Visual Studio code. It had A platform for DevOps back then called Visual Studio Team services nowadays called Azure DevOps. That tools division within the cloud within Azure had the job of providing the ecosystem, the tools, the platform for developers to build on top of Azure, to build on top of Windows, to build on top of other Microsoft platforms. And GitHub snapped into that developer division bringing basically the open source platform, the home of all developers, into that same ecosystem. From a business perspective that meant the GitHub revenue became part of the Azure KPI that is reported in earnings calls every quarter for Microsoft. And we announced last July so almost a year ago that GitHub had passed 2 billion in annual revenue run rate ARR. And you can go back in time 2017, a year before the deal, then GitHub leadership team had announced 200 million million in ARR. So those are not exactly snapping to the timing of the deal, but you get an idea. Between 2017 and 2024 revenue went 10x right? And as such the first thing I want everybody to Think about when you ask, how is GitHub contributing to Azure? Well, it contributes to Azure's revenue number by increasingly generating more and more cloud revenue. And of course Copilot is a big part of that cloud or an increasing part of that cloud revenue. We bring developers, new developers, students, even high schoolers, sometimes even middle schoolers, increasingly younger kids as well. Often the motivation is actually less that they want to learn coding. That is also true. But I was in one of my sons middle school last year and I asked them about GitHub and they're like, yeah, that's why we find an old Minecraft build that works on the school laptops and isn't detected by it. Probably because it runs on some older file that isn't fingerprinted by school it or what have you. But it brings developers into the platform, into the Microsoft ecosystem, and the ultimate hypothesis is that they go through their career journey on GitHub and learn about Microsoft tools, whether it's VS code and visual Studio, whether it's the cloud in Azure. And of course we have within GitHub a number of integration points where you can deploy your repository to Azure and pull something called GitHub Actions, our CI CD service that suggests the Azure Kubernetes service for example, to deploy a Docker file. It's the other way around, which is the Azure team. Teams obviously are also leveraging GitHub for their work, whether it's GitHub itself, whether it's GitHub Copilot, GitHub Actions or the open source on GitHub consuming that contributing back to it. So when we think about developers at Microsoft and developer tools at Microsoft, we always also think Microsoft is one of the biggest software development companies itself. So it benefits from every innovation that we're building into GitHub for its own backlog, for its own shipping, velocity, quality, etc.
- [10:26]
B
The other thing that you mentioned a minute ago that's fascinating is precisely the fact that the acquisition of GitHub seemed to have been so successful in terms of keeping the spirit alive. And to put it in full context, one of the things I find amazing about this whole story is that you guys came up with Copilot way ahead of anybody else in the market. In particular, you were a year before the whole ChatGPT thing unleashed the entire generative AI craze. So you're very much early to this market, which when you think about big companies, that tends to be the other way around, especially for people like me in the venture world is always like the startup moves faster than a big company. That was not the case here. What was the secrets and kind of management tricks and techniques that enable you.
- [11:21]
A
Guys to do that, tying it back to the acquisition. You know, one story I can give you is that beyond all the other reasons I gave you, that we are part of the strategy document that we had to write to send it to Satya and then for Satya to go to the Microsoft board and ultimately get approval for the deal. One additional thing was in fact AI and saying we believe there's a future where we can train AI models on the graph is I think what we called it, of all the source code that is stored on GitHub, but also on the relationships between the developers, you know, how they, how they work together. And so in 2018, obviously we didn't know anything about Copilot, but AI was certainly a reason to do the deal comes 2020. And in the meantime Microsoft had invested into OpenAI. I think that was in mid 2019 and GPT3 was on the horizon and we got early access because of the partnership between Microsoft and OpenAI. And we played for the model, just like today you play with a new model whenever a new one comes out, which feels like it's every other day. And we ask it to write methods like prime number detection, sorting algorithms, those coding exercises, if you will. In fact, through that process we looked at our own coding exercises that we use for our interview loops and I think we collected about 230 or so of them and ran them through the model which then was called Codex. So OpenAI fine tuned a model based on the open Source code on GitHub and other sources. And I think there's a paper still on some of these RIX or whatever the page is called that describes exactly how Codex was trained. I think the model was able to, with multiple attempts to solve more than 90% of these coding exercises, of these 230 coding exercises. And that ultimately gave us the confidence we can build a product here. So this is 20, 22 years and a bit before ChatGPT, but actually conversational coding is what we called it was one of the ideas we had. And the only reason we didn't ship that was that it was good enough and we were quite skeptical. Shipping something where everybody looks at this and asks in the chat interface, describe this meth method or as you know, the typical chat scenario that we are all now used to. And it often gave incomplete answers or wrong answers. And so we felt like we can't really do this and ship this to developers, we're going to get all the ridicule on social media. The other thing that's easy to forget is auto completion has been around forever. Microsoft, I think, shipped IntelliSense, which, you know, it's like the somewhat intelligent auto completion in the 2000 and tens. And in fact even that was debated by developers because the worry was that if it can predict the next word and the method name, I think the term was brainwashed, I'm going to forget my skills. And so that question of is AI going to replace the developer and make us all less, less skillful at lower level of craft? That goes as far back as IntelliSense. And I remember first using TextMate in 2004 or 5 and it felt magical to me that it could autocomplete even stuff that wasn't in the Ruby on Rails app, that wasn't even in the documentation because it would pass the file and create a dictionary of the file I'm working in. And so it could predict the method name that is actually part of the code I wrote. And then Microsoft evolved IntelliSense into IntelliCode, which was IntelliSense with AI using a local model to do better predictions. So that was already AI, it just wasn't a large language model, a transformer model as we used for Copilot. But all these steps ultimately set us up to ship Copilot, the Original Copilot, In June 2021, more than a year and a half before the big bang of AI. And then we brought that into production within a year. And it feels strange, looking back now to summer 2022, three years ago, when Copilot, G8 and how many people were still doubting that this technology is any good and will become a standard tool for developers. And here we are in 2025 and all of that has happened and you see many CEOs in AI and outside of AI predicting these copilots are going to write 90 if not 100% of all code.
- [16:10]
B
Again, in an effort to make this interesting to both a developer coder audience, but a broader audience as well, how would you describe GitHub Copilot in two minutes? And how does that interface with VS code? And what is VS code? If I'm a non developer, what does that all mean?
- [16:26]
A
The way I'd like to start describing what GitHub Copilot does is to describe what a developed job actually is, which is writing code. Most developers, in the morning they go somewhere where they have their backlog, the tasks that they define themselves, or often that comes from their manager or from their product team. They pick up a task and then they go into an editor, which is ultimately a program where I can edit files with plaintext. The key skill of a developer is to take this description that is in human language, because that's what all the specs, all the issues, all the bug reports are in English, German, French, whatever language. The stakeholder used the key job of developers to translate that language into code and to take human language and transfer it into programming language. And so they type code. And as any writer, whether it's a coder or whether it's a journalist or a VC writing a blog post, it's really hard to do that because you're getting stuck when you have an empty file, you're getting stuck because you don't remember specifically how to call a method or how to do a certain algorithm, connecting to some backend system and so on. And so what you did before Copilot would command, tap or alt tab on your computer into a different application, often the browser, and try to find that information. And you do that by opening a new tab and using Google or stack, overflow, GitHub, lots of platforms, Reddit, that have developers talking about these problems and then figuring out what's the answer to the problem I'm trying to solve. And 10, 15, 20 minutes later, you're back in your editor, and you copy and paste that code snippet that you found somewhere, and then you make it work within the file that you were working on. Copilot does all of that without that context switch. It keeps you in the flow. Developers often call this the magic flow state, where you have an idea, you know what you're doing, and it's just flowing from top to bottom in the file, and it feels like you're changing the world, and you want to stay in that state as long as possible. You want to surf the wave, if you will, without falling into the water and having to paddle back and find a new wave. The original Copilot did that by suggesting multiple lines of code instead of just the next word that could complete 5, 10, 20 lines of code. And you would just accept that with a tab key, or you would keep typing until it was good enough for you to say, okay, I. I can now press tab and then edit this a little bit to make it work. Because most developers also are very used to trial and error, trial and error. That's how we all learn coding. And so copilot does that auto completion, and that continues to be the most used features because it is always there. The magic of Copilot in the early days was it wasn't really about AI, it wasn't about large language models. It was about giving you a feature in your editor that keeps you in the flow state and makes you more productive and ultimately more happy. And then came ChatGPT and of course we added the chat scenarios. And that allows you to do all kinds of exploratory things like asking questions, describing code, telling, you know, the chat part to fix a bug. And I bought both my kids a Copilot license because before they would always come to me and say, hey, you.
- [19:50]
B
Had to pay for it.
- [19:51]
A
My Python, of course, that also let me. That also let me test, you know, the payment and the billing flow. And So I get three emails from GitHub every month, one for my own account and then one for each of my sons. And so, I mean, customer zero, as you would call that. And before that they would come and say, hey, I'm building this Python PI game, a little jump and run platform and I have a bug and I can't figure out how to solve that bug. And then I was like, well, you know, I'm talking with Matt right now, but I come to your room in an hour. And of course they don't like that because their stuff is way more important than your podcast. Matt, Priorities, priorities. I bought them Copilot and you know, now they're just asking cop, you know, help me to fix the bug. And it doesn't have to be perfect. I'm not perfect either. When I help them, right, I try to understand what they did. So does Copilot. And So that's the second scenario and then the third one, and that's really what 2025 is all about, is agentic. Agentic scenarios. We call it agent mode in the ide, we have the coding agent on the platform where instead of just asking questions and getting answers, you can give a test and it does that task for you.
- [21:04]
B
You mentioned that GitHub Copilot ran on Codex, which was the early coding model from OpenAI. I think at the end of last year, 2024, you introduced GitHub models, which feels like a library of different underlying models you can use. Walk us through that. How does that work and what can you do as a developer?
- [21:28]
A
In 2024, we did two things. GitHub models was early August and that gives developers access to catalog of models integrated into platform. And in fact, last month, in May 2025, we brought these models into the repository and so you can Integrate these models into your repository to build AI scenarios into your own applications. And that's really cool because you don't have to go to another place and sign up for a new account and then you have your model stuff here and your code on the other side. GitHub ultimately was always about millions of developers collaborating on a project together. And so our philosophy is we bring what we call the primitives that developers need into the repository issues, wikis, pull requests, actions and now models. And then late I think it was late October, we announced multimodal choice for Copilot. Moving from just having one model provider OpenAI to having multiple model providers for copilot chat and for now Copilot Agent mode. And so we added back then Anthropic Claude 3.5 Sonnet and now it's Anthropic Claude Sonnet 4 and Opus 4 and we added Google Gemini back then 1.5, now we are at 2.5 Pro. And what really, you know, it is about choice. We fundamentally at GitHub believe that we need to offer developers choice. Right? You wouldn't be using GitHub if it only had one open source project or one programming language or one ecosystem. Like we are not the ones telling developers us whether jQuery or React or Next JS or whatever the next thing in the JavaScript ecosystem is whether those are the ones to pick. We are offering all these projects at home where they can collaborate and innovate and we let the developers, teams, organizations choose which of these ecosystems they want to participate in or consume. And we believe the same is true for models. There is never going to be one single model that rules them all. But the world of software development is just way too broad for this. And that's also I think where the benchmarks dangerous, I guess because of course there's a big difference between writing code in C C or even Assembler to writing code in TypeScript or Python. And often if when you look at these benchmarks, the devil is in the details. You have to do the double click and see how does it do for my language for a documentation scenario or for test generation scenario. And in fact nowadays in VS Code we not only have the models that we offer on our inference API, we also have bring your own keys so you can connect Copilot to Open Router or Ollama, it can run a local model and try out that model in Copilot to build your own agentic behavior into your developer workflow. So, so that's what it's all about. And that also enables us to move really fast, which has become incredibly important in the AI space. I've never seen anything like that in my 30 year career as a software developer. There's so much innovation, so much change happening that just sticking with one model will ultimately mean you either are really, really good at change management with all these tens of thousands of enterprise customers, that's the hard nut to crack, or you offer the choice and then the enterprises can stick with a model that they feel is the one that is the best for them and approved and reviewed by security and compliance and all that. And then there's all the other models that we can offer other customers and individual developers, those that want to stay at the cutting edge. So that's the other part of the choice being important. If you want to move fast, you ultimately have to support multiple models because there's always going to be a group of customers that, that intentionally want to move slow given the regulated environment they're operating in.
- [25:32]
B
And for the enterprise, do you allow people to fine tune their model based on their data? How does that part work?
- [25:41]
A
We do not today. We have pursued that path in the past. So I think 2023, 2024, a number of companies in our space saw fine tuning as the next, the next opportunity. The challenge from my perspective on fine tun is that A as there's another model every other day, you're ultimately always going to be behind with your fine tuned model and we are putting you then in a position where you're having the choice between the model that you fine tuned that is based on an older version of the base model or you can pick the new model but that isn't fine tuned yet. B I think most customers code bases especially if you, you look into the individual project. Most companies that are at a certain scale have not just a single programming language and a unified stack. That's the dream every CIO CTO is talking about. I want a standardized stack for all my developers and then you look into a 30 person startup and of course even they don't have that because as soon as they go from web development to iPhone development to Android development, they already have three stacks. And so then if you look at the individual repository or set of repositories that could be isn't actually big enough to have a meaningfully fine tuned model. And lastly, and I think this is where fine tuning kind of like got left behind by most companies in the space is that when customers think about fine tuning what they really mean is that the agent understands the code base and the environment that that code base sits in. But that's not only the code that was relevant during the fine tuning process, because you can always have a new open source library being added, or you're upgrading from from one REACT version to another and now your API change or from one Java version to another, even simpler. But it's also everything else that's surrounding that project. Your database, schemas, whatever solution you're using for feature flagging and experimentation. There's all these services that surrounds your project that provide additional context that the agent needs to know about. Now in 2025 I think the answer is very clear, which is MCP Model Context Protocol and the ability for models to call tools. And so they can call tools on the command line to install a Maven package or an NPM package. They can call an MCP server that connects into your GitHub repository or into your JIRA and confluence institutional knowledge. And then there's 1,000 MCP servers for everything else. So if you want to connect to Figma or you want to connect to your Mongo database or what have you, you can put in all that context. So the agent actually knows as much about the environment that the code base sits in as the human developer using MCP using pool calls and then whether it's retrieval or just navigating the file tree and doing code search like a human developer does. And if you have used agent mode or any of these agent modes, you can actually see that working. How it uses the chain of thought to go from okay, it's three files or maybe I need to add two more files. So now it has picked five files that needs to modify and it figures out that it not only needs to write the code, but also the test cases for it, and then run the tests and fix it itself. So this iterative process that the agent does with the help of the tool calls in all the context makes it so much more powerful than a fine tuned model could ever be.
- [29:13]
B
All right, amazing. I'd love now to talk about, let's call it the political economy of this whole space, for lack of a better term. You mentioned the incredible pace of innovation and indeed it's dizzying. Just over the last few days it feels Mistral announced a coding product that's a combination of the various underlying coding models that they had Cursor officially announced they are round at a 9 or 10 billion valuation and said that they were at 500 million in AR, which is kind of insane and makes them possibly the fastest growing startup of all times. At least on the enterprise side. The Sonnet 4 that you mentioned earlier, that came out two weeks ago. It feels like old news, but that was like two weeks ago. So the pace is absolutely dizzying. What you make of it and how you would categorize it, like who does what and where do you guys fit in? What do you cover? What do you not cover?
- [30:15]
A
First of all, I think we should recognize how amazing it is that a developer tools company is the fastest growing startup of all time. Because we're touching times between 2018 and now where that you would go to a startup founder or a venture capitalist and they wouldn't want to invest into that space 100%.
- [30:36]
B
I'm so glad you said that. That was such a VC trope that don't invest in developer 2 because developers are very cheap people. They will never pay any money for anything. And yeah, there you go.
- [30:45]
A
There's no money to be made and here we are. And second, I think it shows that that AI is going to accelerate the role of the professional software developer and it's also going to enable everyone that wants to become a developer to learn coding. And you can see that with Copilot and a bunch of these tools. You know, Vercell with zero bold, lovable OpenAI codecs where folks with a non technical background can build a quick web page now. And I think this is good to anchor ourselves. We have been living in a world for quite a while where companies were pitching no code or low code solutions. In fact, I think 15 years ago I set up my first Squarespace webpage and even though I knew how HTML and CSS works, that was still the faster and cheaper way to just spin up a quick web page and publish. I think it was the landing page for our startup back then. And just say here's the sign up field. Let us know if you want to get informed. When we're launching with AI, we're going a step further because now you can use AI agents to generate quite complex applications from web pages to little games. My favorite demo is the Bull Snake game because it's quick to build and quick to play without explaining too much how it works. Most people still remember Snake on their Nokia PH or whatever phone they had. And so if you look into the space now you have companies that are working in this bootstrapping or we like to call it Greenfield scenario. You start with nothing. You enter a prompt. It's almost like using an image model. You enter a prompt and it generates an image, but instead of an image generates you a Web page, an application, and then you either like what you see or you want to add more or change things and you keep prompting. And in fact the image model is a great I think, comparison because in the same way that the first output of an image model almost never as perfect and exactly what in your head. The same is also true for these bootstrapping AI scenarios. So that's, I think one category one is IDEs. You already mentioned, there's a bunch of others that have taken VS code, forked it and made it a new version. And that's the power of open source. The power of open source, of Microsoft making VS code open Source back in 10 years ago, in 2015. But there's also the traditional IDEs, Visual Studio, the JetBrains family of IDEs, even Apple with Xcode. Everybody is integrating AI scenarios into these IDEs. And many companies like ours are providing copilot not only for VS code and vs, the Microsoft IDEs, but also for JetBrains, for Eclipse, for Xcode, for Android Studio. And then there's forks of our code base to bring it into Emacs and what have you. So the IDE is the hardest playing field, I'd say, simply because that's where the majority of professional developers are. And the key of successful developer tools has always been you got to meet developers where they are and then move them into that future that you're seeing as your North Star. But what doesn't work is to give them the new thing and tell them to stop Everything that we're doing until now that's just going to create an allergic reactions like go away, I know what I'm doing. I'm used to my shortcuts and my color schemes and all these kind of things. Then there's the model companies that provide the models that power all these scenarios. But they have entered the space itself, whether it's OpenAI, Codex, Cloud Code, Mistra code. And I think part of the reason for that is that AI code generation is scenario that worked first and has the largest part of the respected workforce having adopted it. As in most developers have adopted some form of AI tooling into the workflow 30 already, while other knowledge workers, other white collar workers, are still early in that adoption journey. And they might use a chat agent as kind of like a better version of a search engine to find a document or to summarize an email, but they haven't replaced parts of their workflow with AI. Which brings us to the fourth category, which is agents. And these agents, what separates them then from the Bootstrapping greenfield scenarios that these agents work in brownfield, as in they are working within an existing code base for the coding agent or within your Azure cluster to monitor all your resources, the SIE agent, and then look at exceptions or monitoring data and then file a GitHub issue and say, hey, the latest deploy has created an exception and you can tie those together and have the SIE agent file an issue and then the coding agent pick up the issue and provide the pull request. And then the human developer comes back in again to review the code change. What's interesting about these four categories is that while there's startups in each of them, there's also companies like Microsoft and GitHub that operates in all of them and tries to provide a holistic platform across all these scenarios. And I think the key for winning is to have have a continuum where you can have an agent write code for you and submit a pull request. But then if you as a developer see that code and you want to make three quick changes, you got to be able to take that onto a local machine and just make those changes instead of trying to figure out how do I provide now feedback or prompt to describe in natural language, where I already know how to do it in programming language. Right. That's basically replacing something that I can do in three seconds, but something that might potentially take three minutes or even longer. And that's obviously not more productivity, that's less productivity. And so enabling developers to move between those categories and being able to pick the agent that provides the best ROI or do it themselves. I think that's the key for winning in the next few years.
- [36:56]
B
This is so fascinating in so many ways to take your third category of the big AI lab. So there's this competition and collaboration relationship that's starting to appear where obviously Microsoft has a very special relationship with OpenAI. But through GitHub models that we were just discussing, you've diversified. Then you have offered choice for your users, which is the key driver. But as a side matter, you've diversified away from one supplier. That tension also between anthropic, potentially and Cursor, where cursor has started to diversify away from them. So how do you view this evolving? Ultimately, everybody competes but also cooperates. Or is there a moment when, I don't know, you guys maybe need to develop your own model specifically for coding. How do you view it evolving?
- [37:55]
A
It's the DNA of Microsoft to be competing and partnering with many companies in the industry. Right? The. The most classic example comes to mind is that Microsoft for the longest time has sold software or provided software nowadays for the Mac, going back all the way to the Apple ii. I think most of us remember Bill Gates appearing in a Steve Jobs keynote, big on screen behind him to announce that Microsoft is investing into Apple and bringing Internet Explorer and office to macOS. And then of course, you know, after Satya became the CEO, the decision was made to bring the Office suite onto the iPad. And nowadays we take it for granted that Office teams, Outlook, all the modern Microsoft applications run on a Mac and run on iPhone and Android. And as such, I think, you know, and I learned that when I joined Microsoft 10 and a half years ago, it was weird as a startup founder to come come into the company and realizing that what I considered competitors for the product I owned at Microsoft are often also partners. I can't name any names, but many of these AI code generation companies that compete with GitHub Copilot are running their inference on Azure AI Foundry and as such they're paying Microsoft for the lower parts of the stack. And obviously Microsoft's cloud business with Azure partner doesn't want to say to customers, you can't run on us because you're competing with one of our products. We have so many products that would hurt Microsoft's business. So it is part of our DNA to both partner and compete. And I think in many ways that is the incredible thing about the tech ecosystem that that is possible. Meta, AWS, Google, they're also all hosting their open source on GitHub and as such they are in one form or another, GitHub customers. And so we really enjoy being on both sides of that spectrum. And I think it increases the energy that we have within Microsoft and within GitHub to say, okay, we'd love to partner with these companies and developer space even more so, because look, if we just exclude parts of the ecosystem, that means those developers go elsewhere and pay for the subscription or do their business with somebody else in the industry instead. We see ourselves as part of an ecosystem, as GitHub, hopefully being one of the bigger planets, one of the core planets in Star wars, but there's a huge system, Mid RIM and Outer Rim, to stay within the metaphor. And companies that are closer to us and sponsor our conference and have active partnerships and building active integrations even though they're competing. Devon as one such company comes to mind, and others that are farther away but obviously still a part of the developer ecosystem and that integrate with GitHub use GitHub identity or store their source code. And so if you take it back to AI code generation. All these companies ultimately allow a developer to submit a pull request or have the agent submit the pull request onto GitHub. And as such, we have to be partners to enable those customers that pay us money for that repository for their pull request to choose the tools that they want to have part of their platform stack.
- [41:30]
B
I'm curious what it feels like from the inside, running a very important major division as part of the broader ship when it comes to product decisions. A big part of the value proposition of Cursor from the outside is that because they forked VS code, they can have a product that's sort of all in on AI versus very AI native in all aspects of what it does versus GitHub copilot from what I understand, where you're plugin into VS code and then another part of Microsoft needs to make sure that VS code remains very versatile, broadly applicable to lots of different use cases and scenarios. Do you feel constrained by that in terms of how much you can do that's truly AI native because you need to operate in that framework where there are other driving factors?
- [42:25]
A
Not at all. And the VS Code team, while they don't report into me as the GitHub CEO, they are part of the developer division that we are also part of. And together with the developer division, we are part of an organization within Microsoft called Core AI that has the whole AI stack in it as well. And so under Jay Parikh, who's the leader for Core AI, we are forming one unit that competes in in this Azure AI with all these companies across the categories that I lined out and as I mentioned, we're also partnering with them. So a different part of the core AI team might have one of our GitHub competitors as their customers and have partnership meetings with them and negotiate deals and Azure spend and those kind of things Coming back to VS Code at Microsoft Build in May we announced that we are open sourcing the Copilot within VS Code, which also means we are moving it from an extension, which is how Copilot originally started, into a core part of VS Code. And so it becomes open source as part of the VS Code Open source project or the Code OSS project. And as such the VS Code team, and that's been true for a while, is actually the one building the client side of GitHub Copilot. And so for I think the last two years now, now the VS code team owns GitHub Copilot client side for VS code. The Visual Studio team owns the Visual Studio Copilot and then other teams in the developer division own Copilot for Jetbrains, for Eclipse and for xcode. And in fact the XCODE version of Copilot is an open source project from a maintainer that we took over and it started as open source with an MIT license and we did a commercial deal with the maintainer to transfer it into our GitHub organization, but we kept it open source. So the Xcode version of Copilot has been open source for quite a while because it started as an open source project. And we decided that open source maintainer and the contributors to that project did such a good job that it's better for us to pay him some money and take over the project instead of competing with that open source maintainer. I think the key for success at Microsoft is to leverage what is ultimately an incredible machine. And yes, it is a huge ship and so changing direction sometimes feels really hard. But at the same time there's so many incredible people that are working in the developer division, in the cloud division, in the AI division that we don't feel constrained. If anything, I think the hard part in Microsoft is always going to be you have so many developers and so many employees, you know, that are supporting these developers in sales and support and what have you. That saying no to ideas is really hard. That's much easier when you're running a 10 person team because in a 10 person team you know exactly what everybody's doing and you're the 11th man. That also is probably coding if you're the CEO. And so it's really easy to say, well, everybody's busy this week so we don't have time for anything else unless somebody volunteers to work nights on weekends. But if you're managing an organization with I think Microsoft or app has about 70,000 people in research and development, not all of them are engineers. There's some product managers and designers, but the majority are writing code. Saying no to an idea is really hard. And so putting yourself into that mindset, yeah, we are big, but we still have to say no a thousand times before we say yes again. That I think is the constraint that we're having and then figuring out what are the strategic bets and how we are competing in this market, how we're ultimately winning the race and realizing that there's always going to be another race and another race. Right? We're playing the infinite game. As a 50 year old company, yes, we want to win the news cycle every month, the Formula One season of 2025, but then we also want to still be here and relevant in the next decade and more to come. And that I think is ultimately what it is all about. At Microsoft we have always stayed the forefront of technology. We want to keep doing that for as long as we are all here.
- [46:44]
B
You anticipated my question which was what's the bull case and the bear case depending on which idea you look at it from on the cursor side you could say they have the benefit of being constrained and being able to go all in in building one specific product and sort of attracting the kind of developers that already want pure AI. I was curious what the bear case is for them. Like the reason why they're be a roadkill from the power of Microsoft and perhaps Google. So that's distribution, that's developer mindshare. You're a very competitive guy and Microsoft is a very competitive organization. How are you going to crush them?
- [47:28]
A
I'm not thinking about crushing them because I think that's not how we are seeing the space. I like for developers to pick the tool they love most and I like to build the ones that most of them are picking. And so I think realizing that in developer tools and in ultimately in technology there's always going to be multiple players. There's always. And some of them and some technologies stay around forever. Cobo and Mainframes is one such example that comes up way more often than you would think for me as GitHub CEO, because many companies still deal with that COBOL Mainframe. Now is that you know, the state of the art and something that college kids get get excited about? Of course not. But we should realize that once some software is deployed it sticks around. It's incredibly sticky and it sticks around forever. And migrating things, migrating even a repository in our form subversion into into Git and then into onto GitHub that are often very complicated and expensive projects that engineering leadership only wants to fund if they can see the ally. I think the back case for any company in our space is missing out on the Next Big Thing. And there's always going to be a Next Big thing in software development, whether it's front end frameworks, whether it's programming languages, whether it's IDEs. Maybe the IDE is going to be replaced by something much more web based and you can in fact already do that by Copilot and assign an issue to the coding agent on your iPhone with the GitHub mobile app and then it does its work while you're maybe on a podcast and then it thinks you and says here the Coding agent has submitted a pull request and then you can review that code and you can even do that on your iPhone because reviewing code is fairly doable on a small screen. Writing is harder mostly because of all the special characters and what have you, predicting what is the next big thing and then being able to disrupt yourself. Overcoming the innovators dilemma that ultimately every company will get into that place where they have a moat, where they have an existing revenue stream stream even more. So that revenue stream comes from many enterprise customers to then say okay we want to preserve that cash flow but we also realize we got to disrupt ourselves and move into the next version. And you have heard many of the startup founders, CEOs, chief product officers of our competitors talking about that likely the world of AI co generation and IDEs will look very different in a year or two from now. And I think we all realize that and, and so both the bear and the bull case are actually very similar which is can you predict what that looks like and meet developers expectations and have actually the user adoption that then justifies that disruption to your existing business? And at Microsoft that is given the size of Microsoft, that is both an easier and a harder task at the same time.
- [50:27]
B
That's such a fantastic thought that the pace of progress actually creates an innovators dilemma for everyone. Not just for Microsoft, but actually probably for Curso. As they grow they will also have a business to defend given how quick things are changing. That's such an interesting thought.
- [50:44]
A
GitHub has been pooled at least twice, right? GitHub was founded on Git, which wasn't invented by GitHub, it was invented by Linux, Torvalds and the Linux kernel team. And a few years later, two years I think the GitHub founders, Chris, PJ and Tom saw an opportunity to say Git hosting is so hard, let's build you know, a SaaS service. And in 2007, 2008 you know that was a time when SaaS wasn't as established, certainly not cloud based as it is now. And then we did this again with Copilot where we again took pick the technology that wasn't invented by us. Neither Transformer nor the GPT model nor the inference infrastructure came from us and they came from OpenAI and paper, obviously from Google and the infrastructure for Microsoft. And then we built the first, first the first Copilot and the first successful product in that space. And so yeah the big question is what's the next wave of that? And we believe it's coding agents and that's why we are investing heavily into our own coding agent in Copilot and getting that from the 2030% benchmarks. If you would look at multilingual or 60 something percent for just Python getting that to actually a number, I think 90% is going to be the minbar for broad adoption and saying this is now established technology and we need to look for the next big thing. That is going to be the challenge of the years to come.
- [52:12]
B
Yeah, let's double click on that for a few minutes. The launch of that agent mode was a major announcement, major new step in the history of GitHub that just happened at build 2025 a few weeks ago. What does it do and what tasks in particular would you suggest people should direct directed it to?
- [52:32]
A
So the coding agent, the way it works is that you can just give it a task, coding task, documentation, generating test cases, or simple things like find all the bugs in my code base. The agent then, you know, goes off in the cloud and spins up a virtual machine and checks out the repository, installs other tools and solves that task for you. And the magic here is that in the meantime, meantime you can keep working on your part of the code base on a different task, a different issue on your local machine. And so effectively the coding agent is like a new member of your team that can take on certain tasks and you can obviously not only assign one task to one coding agent, you can assign 10 tasks to 10 versions of that coding agent and they can all run in parallel. And when they're done done with their work, they submit a pull request exactly like one of your human team members would do. And then they alert you and say, hey, this pull request is ready for review. And then you go in and you review the code and you can comment on it and the copilot will pick up those comments and keep iterating. And so if you don't like the code or. I tested it yesterday with one of my hobby projects and I realized the readme is completely outdated because I didn't spend time on writing a readme for myself and just told the coding agent, look at the code code base and write a new readme. And then wrote a readme. And I reviewed the readme and looked at the headline, it changed the title of the readme from readme to something else. And so I gave it feedback and said, hey, I think the readme should always be called readme and so revert that change. And it just did that, right? And so that's the most simple explanation that I can give of the coding agent. There is nuance here because there are going to be scenarios where now you want to take that code generated by the coding age agent and check it out on your local machine. And you can do that with the GitHub command line and then keep working in Vs code. And of course, copilot agent mode is also available for you there. And so you can then use the agent in synchronous mode on the local machine. And so that's if you stay with the analogy of your team, that's like you pulled that team member that built the pull request onto your desk or into a zoom call and now you're working together on that code base. But. But now your focus is on that task in synchronous mode while the coding agent works asynchronously. And so this continuous spectrum of I can assign tasks, task generation, bug fixes, security vulnerabilities, bootstrapping a new feature to the coding agent to I can take that code base into my local IDE or I can keep working just as I'm used to all these things in parallel. That I think is going to be the future of software development. And then the skill of the developer will be to know how to describe the task in such a way that the agent can do the job with almost no additional revisions needed. Because every time you have to review code and correct the agent and wait another 10 minutes, you're interrupting your flow again and you have to do something else in those 10 minutes. And jumping between different we're bad at multitasking and we're bad at context switching. And most developers do not want to work work on 30 different things throughout the day. That's stressful and ultimately doesn't lead to great products.
- [55:50]
B
And the agent works with prompts, right? It's vibe coding where you describe what it is that you want it to do and it does it. Is that right?
- [55:58]
A
It's prompts when you use it within the ide, although even there you can start with brainstorming cycle first. And so for example, that works really great with cloudsonnet or Cloud opusq for and so you can first ask it, you know, how would I build this? And what are the. What's the system design for this feature, for example? And then have it first write a markdown file with bullets doing the engineering together with the model. And then you take the first task of that and feed that into the agent mode to write the code for the coding agent. Because it sits on GitHub platform, the starting point is an issue and the issue can be the description that comes from a product manager or from a user of your open source product project. But it's also all the comments in the issue, attached images, file references or web pages. And of course the coding agent and agent mode both can use MCP servers and tools and so you can then further connect into additional context. But yeah, fundamentally it's a prompt, it's just that the prompt, it's no longer just one input field with three lines of code. It can be a long description specification from a product manager, just as they would write for for the human developer. And again, the product manager then is the one that needs to learn of how much do I need to decompose the big problem into a small building block?
- [57:23]
B
And when you mention 30% or 40% success, is that across the board or from your experience using the product and early feedback from users, is the agent particularly good at certain tasks versus other tasks? If I'm a GitHub user and I want to play with product, what should.
- [57:43]
A
I do first, the 30% I was referencing the Z bench swe benchmark that was developed by a number of researchers and originally started as in Python only. And it's 2000 issue pull request pairs out of a dozen Python open source repositories, but was recently expanded to other languages. And so in Python I think the big best benchmarks is 60, 60 to 70% depending what model agent combination you take. But if you look at multilingual, we are in the 20 to 30% range for these benchmarks. So that gives you an idea of real life issues in open source projects and the corresponding pull request. How good is the agent on solving that existing issue compared to the original solution for our quoting agent, the way to approach this for developers to just go and try it out, because worst case scenario here is that you get a pull request that is so far off from what you would build yourself that you close the pull request without merging it. But then that gave you a learning cycle between the description that you gave it and the code it generated. And then there's multiple ways you can approach that if you don't want to throw it away. One is to provide custom instructions in a file within the repository to give the agent more context of what you expect it to do. So it's kind of like a how to that you provide to the agent in the same way that if you would hire a developer into a team or you would instruct people in open source to contribute back to your project, you will also give them a this is how we expect, you know, the coding standards, the libraries, how we're writing unit tests. So giving the agent more instructions often leads to better outcomes. Scoping it down to a smaller change often leads to better outcome and also to smaller pull requests, which is the best practice. And over time, the models will get better. We launched the coding agent on Cloudsonnet 3.7, and then three days later, Anthropic came with Cloudsonnet 4. And so now the coding agent is running on the full model. And of course, you know, these models will get better, the context for the model will get better, the tool calls will be better. And so I think the key thing to always remember is whatever your experience was with AI months ago is probably no longer the state of the art now. You shouldn't really form strong opinions or. Or if you have strong opinions, you should have them loosely held and change your beliefs. As AI technology is getting better to.
- [60:12]
B
Close, it feels almost inevitable when having a conversation about AI coding in 2025, to not ask about what that means for the broader industry and the broader world. So two questions. What does that mean in terms of the future of SaaS in a world where anyone can spin up applications quickly? Do we all build our own applications ultimately? And then another big, meaty question, which I'm not suggesting we go into it in great detail, but what does that mean in terms of the future of being a coder? What are the skills I need to develop or not care about in the.
- [60:55]
A
Future, the future of coders or software developers? Software engineers is probably the best way of framing. It is bright from my perspective, and software developers will learn very fast, if they haven't already, to adopt AI within their workflows. Not only Copilot and not only in the IDE or on GitHub, but also on the command line, when processing files, when organizing files, all the things that we also, when operating clusters, all these things that we also do. And we have always, as software developers, automated parts of our workflow and moved higher up the abstract. When I started coding on a Commodore 64, there wasn't a debugger. Everything was trial and error, and you would print stuff onto the console to see what the variable content is. And then came Turbo 4 Scale and Visual Studio and so on. And our tools got more complex, and as such, we moved up and got more done in the same amount of time. And I think for developers, the same old thing will happen again. It's in fact already happening. It happened with the cloud, it happened with containers like Docker and kubernetes. Today, most developers in the web don't really care about the hardware anymore, certainly not about the chip itself, other than, you know, in models where you, you might know that you're running on an Nvidia A100 or H200 or whatever the latest generation is when you're listening to this podcast. But you know, for most cloud applications, what's actually below, below the application layer doesn't matter to the device developer as long as the service scales and is fast and snappy for the users. Which brings me to the SaaS question. And I think, you know, everything that I can easily replace with a single prompt is not going to have any value. It will have the value of that prompt and the inference in the tokens, but that's often a few dollars. And to manage the allowance of my kids, I'm probably not going to pay a $10 SaaS service. And I have a hard time seeing what additional value they can provide instead of just using, for example GitHub Spark, which is our tool for these kind of scenarios and build my own application. But at the same time I think existing SaaS services will also increase their complexity. And so as the AI technology is not only used by you and I to bootstrap application, it's also used by those SaaS companies that are building software like GitHub itself. We are making our applications more complex and add value you that hopefully still motivates customers to Pay for the SaaS service. And whether it's for GitHub Copilot or for the fastest growing startup or the previous fastest growing startup, which I think was viz, I think there's always going to be companies that use the latest technology to build products that you cannot easily replicate yourself. And as such I'm not worried about, about that that value chain goes away. I think it's just, I think we're underestimating how much you can do with the models that are out today, not the ones that are coming next year or next month, the ones that we have today where I think we're only touching the surface of what's possible with those leasing capabilities. And there's a whole new generation of products and agents that will, you know, offer scenarios that, that we're easily willing to pay a SaaS price or a mix of seed based pricing and consumption.
- [64:19]
B
Based price pricing for Thomas, what a fantastic conversation. Thank you so much for doing it.
- [64:24]
A
Thank you.
- [64:26]
B
Hi, it's Matt Turk again. Thanks for listening to this episode of the MAD podcast. If you enjoyed it, we'd be very grateful if you would consider subscribing if you haven't already, or leaving a positive review or comment on whichever platform you're watching this or listening to this episode from. This really helps us build a podcast and get great guests. Thanks and see you at the next episode.