← The a16z Show
The a16z Show
Building APIs for Developers and AI Agents
00:26:34
Stainless founder Alex Rattray joins a16z partner Jennifer Li to talk about the future of APIs, SDKs, and the rise of MCP (Model Context Protocol). Drawing on his experience at Stripe—where he helped redesign API docs and built code-generation systems—Alex explains why the SDK is the API for most developers, and why high-quality, idiomatic libraries are essential not just for humans, but now for AI agents as well.
Loading summary
Transcript24 lines
- [00:00]
Alex Rattrade
So I love to say that APIs are the dendrites of the Internet. So if you think about the brain being a collection of neurons that by firing together, you know, think you can think of the Internet as a collection of servers or programs running and running on servers that by interacting with each other, think or do, and that connection, the synapse, that's almost tautologically, almost always an API of some kind. Realize that every company is becoming an API company and every API company needs an API platform with SDKs and docs and API versioning and just a million different things. That amazing team that I worked on at Stripe all collaborated on. People want to be able to focus on their core capabilities, be thoughtful in exposing the interfaces to users and not have to worry about the low level details.
- [00:50]
Podcast Host
If you've been building with APIs, you know, they're the connective tissue of modern software, powering everything from payments to cloud services. But as APIs multiply, so do the challenges developers experience. SDK quality, documentation, versioning, and now a new frontier. Interfaces not just for humans, but for AI agents. In this episode, Stainless founder Alex Rattrade joins A16Z partner Jennifer Lee to share how his helmet Stripe shapes Stainless mission to bring world class developer platforms, SDKs, docs and more t every API company. They discuss why SDK is often the API for developers, how MCP reframes APIs as interfaces for large language models, and the new design challenges of treating AI agents as first class users. It's a conversation about the future of APIs agent experience and what it means to build developer platforms that are robust, idiomatic and built to scale. Let's get into it.
- [01:50]
Alex Rattrade
So I started working on API tooling when I was at Stripe. I had been working on other teams and I just saw how important the API platform was and so I moved over to that team and built out a redesign of the Stripe API docs, which was an incredible experience. And through that saw how important the SDKs were even internally at Stripe. People I think didn't quite see that to the developers who use the Stripe API every day, the SDK is the API. And those were kind of falling behind. So I built a code generation system there that can take this ripe open API spec and turn it into a great Python library and a great typescript library and so on. And that was really fun to build. You know, the computer scientist in me just had a phenomenal time building sort of the compiler aspects of that and the library design nitty gritties. And so after I left Stripe, people Kind of kept asking, hey, you know, how do I get good SDKs? Or how do I build a great developer experience for my API? And it was very painful for me actually, because you know, you're saying, hey, that's a great question, you really need SDKs. It's a core baseline part of the developer experience. At this point, everybody interacting with your API is going to expect a well typed SDK in their language. Unfortunately, the bad answer to that great question is that there's no way to do it. You know, the open API generator that's open source at the time certainly was pretty far from the Stripe quality bar in terms of the output it would generate. A lot of great work has done gone into that, but it's a very hard problem. The maintenance burden, the headaches involved in trying to set this up, up, it's just too much for almost any company other than, you know, something like a stripe at 2,000 people, which was when I worked on that. And so it's very dissatisfying as an engineer to tell people a bad answer like you need this thing and also you can't have it. And so eventually I started saying, hey, would you pay real money for this if I built it for you? People both said hey, yes I would. And also, can you help us with everything else around our API? And realized that every company is becoming an API company and every API company needs an API platform with SDKs and docs and API versioning and just, you know, a million different things. That amazing team that I worked on at Stripe all collaborated on and so, you know, can we bring that Stripe developer platform in a box and bring it to every company? And so that's sort of the way that I came to start Stainless API as a company.
- [04:00]
Jennifer Lee
That prediction you had or that statement you had for 2022 definitely turned out to be true. Today every company is API company and we're going to dive into MCPS too. Can you explain sort of how did you approach the sort of SDK generation aspect of the product? And Stripe experience of course is very critical to building this product is till today is still one of the north stars of what developers look to as great developer experiences and great SDKs?
- [04:26]
Alex Rattrade
Yeah, we had a lot of experience with this at Stripe and you know, saw a lot of the good and a lot of the bad and a lot of the ugly. There's, there's so many details to get right, whether it's making a client library that's really robust and when things get weird on the Internet, your SDK does not. It's not the thing that's blowing up or causing problems. It can handle scale, it can handle great error messages and can work well with telemetry and all this stuff. In addition to being robust, an SDK also needs to be really polished. And that means if you're building something for a Python developer, that it's idiomatic to them, it's familiar. If they hover over on a thing, there is documentation that pops up in their editor telling them about that thing, whether it's coming from the REST API or from the core of the library. And crafting that is really quite a journey. You know, it's something that I just personally love. Years ago, I created a small programming language and it was all about kind of polishing those nitty gritties and finding the best way to do every little detail. And so that's one of the things that I most enjoy at Stainless. There's an interesting thing here where, you know, we're going to be talking about MCP and crafting. I think the Python library that we do is a great example of this, because Python was the first programming language that I learned, no offense, Visual Basic. It had evolved a lot since I had last used it when I was sitting down to create our Python generator. There's types in Python and there's multiple type systems, there's Async, there's, you know, pattern matching. It's really incredible to see. And what's idiomatic through all of this world, and that's a moving target. I think the industry spent years basically figuring out how to make an enigmatic Python library and figuring out what it looks like for a Python library to be intuitive and legible to a pythonista. And that's continued to evolve, and I think that we've moved the industry forward a little bit, and I'm proud of the work that we've done there. When you come to mcp, which is this kind of new frontier where instead of exposing your API to a Python developer writing Python software, one of the things that you can do with MCP is expose your API to an LLM agent using an MCP client. In that world, what's intuitive to an LLM, what is usable and ergonomic and idiomatic to a large language model, to an AI in interacting with a large API. And the world is starting from scratch there almost. So there's a lot of interesting work to do and develop and figure out. So for us, it's kind of this really fun exploratory research problem.
- [06:46]
Jennifer Lee
Maybe Zoom out a little bit. You called APIs being the dendrite of the Internet, now that MCPs are everywhere, like you know, occupying a lot of the brain bandwidth for developer tooling companies like Stainless. What do you call mcps? And how did that have impacted your business, how you think about the landscape?
- [07:03]
Alex Rattrade
Yeah. So I love to say that APIs are the dendrites of the Internet. So if you think about the brain being a collection of neurons that by firing together, you know, think, you can think of the Internet as a collection of servers or programs running and running on servers that by interacting with each other, sort of think or do. And that connection, the synapse, that's almost tautologically, almost always an API of some kind. So it's a really critical part of the Internet software ecosystem. And seeing that kind of like CPU brain, you know, extend into the GPU brain is a really exciting moment, I think. And you need the right interface. So MCP kind of is that interface. I don't know if I have a biological metaphor off the top of my head for that. I kind of want to compare it to eyes or ears or a new sense. When a software system is communicating internally. Okay, cool. But it's all in your head in some sense. It's all in the cloud. It's not doing anything in the real world. And how does you know all the software that the technology industry has been building for the last 30 and beyond years, how does that kind of become real in the real world and do something meaningful? And one of the long standing ways of doing that is sort of, I'm going to call it through the eyes in the form of user interfaces. So you'll have someone log into a web application. You know, I used to work at Stripe and so I use this example a lot. Someone at a small business might log into their Stripe dashboard, refund a customer, create a new payment, add a customer to the system, anything like that. In the days before APIs, everything was this way. You know, then we saw that we needed to automate things, you know, maybe instead of eyes, I don't know. Now we're going with fingers. Right. And there's some part of your, you know, where there's a communication between your mind and the neurons and something tactical in the real world where all of a sudden these computers can do things in an automated fashion. I don't know if you would call MCP and AI like they're the new ears or something like that, but it's a new sense or it's a new way of interacting with, getting information from the world and also putting something out. You're sort of like growing this new limb.
- [09:09]
Jennifer Lee
I definitely imagine a hand raising up in the air of like, here you can use me and like, I can do these things. So definitely, you know, having more of the touch points and surface areas for AI to explore and also to leverage a lot of the existing, you know, capabilities from software. So I like that analogy a lot. And just maybe going down that train of thought, you have been developing for tools for developers and as a developer for a long time, but now we have this new Persona that's large language models and AI and they are part of the design aspect of how to serve large language models as a user and building sort of tools. Thinking about this new Persona, what are the changes and challenges in just the thinking of designing tools when taking in this sort of new interface and new requirements that has impacted how, you know, you're being building and building the products?
- [10:03]
Alex Rattrade
Yeah, so if you think about a Python library being the interface between an API and a Python program or Python developer, I would think about MCP as being the interface between an API and an LLM. And so to us it's all SDKs right now, again, you can do other things with mcp and these other things are important. Documentation is really a big example and I'd love to talk about that too. But in terms of sort of these, like core operational, you know, let me go refund a customer in Stripe Dashboard or let me go find out when this last transaction occurred or anything like that, you know, as you say, there's a lot of difficulties and new challenges and problems in making this work. And a Python developer might really want to make sure that Async works without too much rigmarole, or a Java developer might want to see that the classes that they're interacting with are type safe, but at the same time may not want them to sort of blow up at runtime if something is unexpectedly null. And that's a really hard problem. Right. You know, yesterday I was chatting with someone who was on the Java team back in the day and helped design some Java languages. And I was telling him about this, he was like, yeah, that's an unsolvable problem. Right. And, you know, that's something that maybe solved is the wrong answer, but that we've tackled at Stainless. And I'm proud of. There's really difficult challenges for elements. And so some of those challenges that you're asking about, a lot of it comes down to the context size. And so if right now, with mcp, that the naive way, the typical way of taking an API usually described with an open API spec and presenting that to an NLM through MCP is you basically take the JSON schema for every single endpoint, every single operation in your API. You turn that over into a tool in your MCP server and say, if you want to create a charge in Stripe, you know, you use this create charge tool and inside of that tool, here's every request parameter, so here's how you specify the amount of the charge, here's how you specify the description, the currency, so on and so forth. And that's going to have the same descriptions and documentation that you would see in Stripe API docs. That's all going to be there in the tool description. You can get accuracy there. But if you want to describe every single endpoint in the Stripe API with every single parameter fully documented, congratulations. You've just used up your entire context window. And that basically just doesn't work. Even if you don't use up the whole thing. LLMs tend to kind of get overwhelmed. MCP clients tend to kind of get overwhelmed at seeing too many tools in use or available. And all these request parameters, like you may not need them. You know, maybe you're only trying to. Maybe you only need three parameters from this endpoint, not all 50. Or maybe you only need one or two endpoints from this API, not all 500 or 50 or what have you. So that's a really difficult challenge to figure out how to gradually unveil what are the tools available, what are the parameters available, what are the descriptions of those parameters? And you want to be able to do that without too much back and forth, because every turn takes time and takes tokens. Tokens, yeah, exactly.
- [13:01]
Jennifer Lee
Yeah. So do we have a solution for the context window challenge as well as the limited tool calls? Is there a solution yet?
- [13:08]
Alex Rattrade
There's several things that we do to help with this today as stainless, and there's several things that we're doing soon. But the most simple thing that we do today is we can generate all of the tools in your API. And when someone wants to spin up this server, they can use command line flags in the CLI or query parameters in the remote URL to basically say, hey, I only want to interact with these resources. So I only give me the, you know, payment intents part of the Stripe API, only give me the customer's part, the Stripe API. Or I only want reads or I only want writes. And that limits things down a lot. And the end User can decide what they want to do, the API provider can decide what they want to do. And that nipping and tucking is really useful at sort of a first approximation that could be a little bit clunky. And so maybe you don't know in advance what the limits you want to do are. And so a more dynamic approach that we also provide is the ability to say, hey, give me this in dynamic mode. And instead of say, you know, 50 or 500 different tools, we only expose three, right? So you know, it scales O of one, not O of n. And you have a tool to get the list of endpoints available, the list of operations available. You have a tool to describe a single endpoint and get all those request parameters. And you have a tool to execute an endpoint. And so you've gone and made these tool definitions totally dynamic. Now there's some trade offs there. It takes three turns to do one operation here instead of just one. And so at runtime that's a bit slower, but it saves context window. And if you end up not interacting with, you know, again, say the Stripe API, then you don't burn any of your, you know, hardly any of your context at all, which is terrific. So that's another thing that we offer today. A third thing that we do currently I think is really fun and interesting. This is something that we shipped recently. It actually solves a slightly different problem, which is, okay, let's say you make your API request and you say, hey, Stripe, you know, tell me about the customer Jennifer, right? And it'll make a retrieve request to the customer endpoint, it'll find Jennifer or actually a list request, it'll come back with a bunch of data right now, especially if it's a list request. So maybe you make a request to Stripe API, it tries to find the right Jennifer. There's a lot of Jennifers in there. So it's going to have to sort it through. And the size of the API responses here can be very large. In fact, especially for like a list request like this, they'll typically be so large that your MCP client like, you know, cursor or claude code or anything like this will just refuse to look at it at all. You know, it does. You just literally can't interact with it. Even if it can, it's just too much data to sort through everything. What we've done is we've added sort of this JQ filter to all of these requests. And so JQ stands for JSON query and it's a commonly used clique for Saying, hey, out of this large JSON object, give me only these properties and filtering down the data that you're getting. So sort of like the select clause in SQL for those who are familiar with that. LLMs are great at JQ, so just giving them that tool has enabled them to say, out of this large API request, I only need Jennifer's like name and email and description, and I don't need to know all these other details. And that keeps the, again, the impact on the conto context window light and it keeps things really focused.
- [16:29]
Jennifer Lee
That's incredible. That sounds really useful. It reminds me another question of there's a misconception or misconceived notion that now you have MCPS and that's how LLMs are accessing and interacting with APIs. The quality or interface of APIs or SDKs are not that important anymore. But I think that's like the polar opposite. It's exactly because there's more usage. You need strongly typed, very polished SDKs as well. Do you mind elaborating on that?
- [16:57]
Alex Rattrade
Yeah, absolutely. You know, so I was chatting about this morning actually with someone who leads developer relations at a large financial technology company. She was talking about this problem where today, if you ask a coding agent to integrate with this company's REST API, the first thing the coding agent is going to do is install this company's SDK. That sounds great, but what she's seen is that it's going to install a version from 10 months ago, an old version, and then it's going to hallucinate the whole interface. Right? And I think a lot of people have had this experience where they say, hey, ChatGPT, can you help me integrate with ChatGPT? And it gets the version of the SDK a little bit wrong, right? And so something that we need to see with mcp, and this is sort of, you know, something I alluded to earlier, where the other big use case for MCP, rather than interacting with an API, is writing code and helping a coding agent basically access documentation. And so something I'm really excited for us to ship very shortly is the ability to get comprehensive reference documentation for the API of your library into the hands of an LLM, so that, you know, when people integrating with this financial technology company say, you know, hey, make me an integration, it'll know the right version and it'll have access to the full API of that version's SDK and it won't make mistakes. So that's one of the things that I'm really excited about all this Kind of coming full circle. And I'll note, you know, again, there's still a reason that the. It sounds bad that the, that the agent, the coding agent, kind of got the SDK usage wrong in the first place. But it's important to note that the agent preferred to use the SDK. It didn't want to just make a fetch request given the option. And the reason is that with the SDK you have type checking. So if you have an LLM that's kind of like trying to one shot, a whole big crazy API request that's not so easy to do with a hundred percent accuracy, you really want those red squiggly lines saying, hey, this property has a typo or hey, you're assuming that this string is always going to be string and it's never going to be null in the response. You know, a human reviewer is just not going to catch a mistake like that. For an untyped interface, they need to see their type checker kind of giving the quick thumbs up that the LLM is in the right direction so that they can basically quickly review the business logic that was produced by the coding agent. You know, I think a lot of people are seeing this, that the more declarative and clean and dry the code that you're writing and the more type safe the code that you're writing is, the more you can trust LLMs to do the coding part of your job for you and think about the higher level stuff and keep an appropriate eye on things.
- [19:40]
Jennifer Lee
Totally. So it's not just about keeping the accuracy of the LLM outputs. I mean, certainly helps with that when you have a more constrained exploration space, but also just debugability. Like it's so much better when you have strongly typed SDKs and very, I would call it, like you have the tools but also have the reference docs and menus in place with the docs aspect of it. Like it's going to be a more holistic experience given to the AI agent to really perform the task.
- [20:08]
Alex Rattrade
Exactly. Your coding agent is going to be able to try something out, press, save, see if there's a type error, and iterate by itself automatically. Rather than trying to make an API request, it'll need an API key for that. It might be a destructive API request. Having an LLM test in production is with a foreign API, with a foreign system, that is not what you want to be doing. Right. And so, you know, for any company that ships an API that they want their users to be using in production, whether it's a Big business user, you know, an enterprise doing a key integration or whether it's a developer trying to move quickly. In either case, it's really critical to make it easy for people to move fast at that integration so that it can happen before some other project comes in and for them to do that with confidence.
- [20:55]
Jennifer Lee
I love to just think about maybe the role of API Developers. In five to 10 years you have been a developer also develop really well known APIs, SDKs as well. If we imagine the future where the agents are going to be as common as human developers and we are going to have this agent experience instead of just dev experience, what do you think that experience is going to be looking like down the road? Today we're still continuing designing for both of these two Personas or users. I still call it like LLM a user. What do you think it'll look like in that time horizon?
- [21:33]
Alex Rattrade
I mean, I think it's a, it's a really interesting and exciting question. What I was saying a minute ago, you know, where an LLM, you know, coding agent, writing your code means you want your code to be more declarative and more dry. You know, you've always wanted that. But it becomes more important to be able to quickly and accurately review code. And so for people building APIs, you want a way to write your backend code that is going to be not like spaghetti. And that's how a lot of people unfortunately have to write their rest APIs today. And furthermore, you're going to want more clear and prescriptive standards around what API design should look like so that the LLM can follow a set of rules that the company has put forward around how they want to do their API design. And again the LLM can follow the slot into maybe the API framework that the company is using and just focus on, okay, let's declare the endpoint, let's declare the parameters, let's write the business logic so a human can quickly review that.
- [22:35]
Jennifer Lee
I'm imagining really high taste developers, you kind of crafting the endpoints, how you're designing both the APIs themselves to the documentation with it and SDKs in a very detail oriented way. But now we have a lot of, let's say Cogen tools and we have even more advanced frameworks and also much higher level tooling that you can define the specs. Maybe I wouldn't call it natural language, but just more of like a spec out way but not having to worry too much of the polishing on the details. And that's exactly what I feel like Synolicit's work is doing is to maybe elevate the developers to think about more of the higher level tasks because of what you want the functions to be. And we want the users to interface with instead of like caring about the errors and the details and pagination or like debuggability, observability, these type of nuances.
- [23:27]
Alex Rattrade
The most painful and difficult part around building APIs, there's many painful and difficult parts, unfortunately today for people who are shipping APIs to their customers and their users and their partners, which is unfortunate, but one of the worst is sort of the design part, you know, and dealing with all the bike sheds and thinking, you know, you just want to be able to say we need an API that does this thing right, that manages this kind of resource. You know, the humans should be thinking about, as you say, the high levels of what the business needs and letting the robots do all the less exciting stuff. Now you want AI to be writing your maybe business logic. You don't really want it to be writing your platform code, those core things that sort of operate as all the glue between everything else for that. You want a good platform, so you want a good framework, you want good libraries. And that's something that Stainless is looking to bring to everything around the API so that the humans can say the high level stuff, the LLMs can build that media layer and that Stainless can take care of the low level infrastructure.
- [24:24]
Jennifer Lee
The last question I want to ask is you work with some of the most cutting edge logos and customers like OpenAI, Anthropic, Cloudflare, without disclosing any confidentials. Is there any learnings that's worth sharing with the audience on these collaborations of how they're thinking about their SDK design, their API platforms and what they want to offer to developers?
- [24:46]
Alex Rattrade
Yeah, I mean, I think there's so many lessons that every API team learns again and again. Everyone in this space sees how important it is to be thoughtful, how important it is to consider the user. The exact way you consider that is changing a little bit. Just like it did when people moved from writing JavaScript to writing TypeScript, or when people went from having slow connections to fast connections on the Internet. The fundamentals, I think are still fairly universal. It's exciting to see people really think deeply about how to expose their capabilities. Whether those capabilities are a large language model that can do all sorts of, you know, text and audio, or whether that's the ability to spin up new servers in the cloud instantly, or whether that's the ability to move money around the world. People want to be able to focus on their core capabilities, be thoughtful in exposing the interfaces to users and not have to worry about the low level details.
- [25:40]
Jennifer Lee
Thank you so much Alex. This is wonderful.
- [25:43]
Alex Rattrade
Thank you Jennifer. This has been fun.
- [25:47]
Podcast Host
As APIs evolve into core infrastructure for both developers and AI agents, The quality of SDKs, docs and interfaces will only grow in importance. If you enjoyed this conversation, please rate and review the podcast on your platform of choice. Share it with your network. Stay tuned. We've got more great discussions on the future of software, infrastructure and AI coming soon. As a reminder, please note that the content here is for informational purposes only, should not be taken as legal, business, tax or investment advice or be used to evaluate any investment or security is not directed at any investors or potential investors at any A16Z fund. For more details please see a16z.com disclosures.