← Software Engineering Daily
Software Engineering Daily
Vercel’s Developer Frameworks with Ary Khandelwal and Max Leiter
00:52:25
The availability of high-quality AI model APIs has drastically lowered the barriers developing AI applications. These tools abstract away complex tasks such as model deployment, scaling, data retrieval, natural language processing,
Loading summary
Transcript101 lines
- [00:00]
Kevin Ball
The availability of high quality AI model APIs has drastically lowered the barriers to developing AI applications. These tools abstract away complex tasks such as model deployment, scaling, data retrieval, natural language processing and text generation. Vercel has developed a complementary set of tools for building AI web applications, including their AI SDK v0 and the Shad CN UI component framework. ARI Kandelwal and Max Slyder are on the AI team at Vercel. In this episode they join Kevin Ball to Talk about the AI SDK v0, SHAD CNUI and the AI tooling ecosystem at Vercel. Kevin Ball, or K. Ball, is the Vice President of Engineering at Mento and an independent coach for engineers and engineering leaders. He co founded and served as CTO for two companies, founded the San Diego JavaScript Meetup and organizes the AI in Action discussion group through latent Space. Check out the show notes to follow K. Ball on Twitter or LinkedIn or visit his website K Ball LLC.
- [01:18]
ARI Kandelwal
Hey guys, welcome to the show.
- [01:21]
Max Slyder
Thank you. Happy to be here.
- [01:22]
ARI Kandelwal
So excited.
- [01:23]
Yeah. So let's maybe start out with some intros. So actually I'm going to go to ARI first. Can you introduce yourself a little bit about your background?
- [01:31]
Yeah, My name is Ari. I was a computer science student at Princeton and after graduating I worked on a design to code startup that Vercel acquired around a year ago. And after that I started working on the V0 team and I do product and engineering work there.
- [01:46]
How about you, Max?
- [01:47]
Max Slyder
I'm Max. I'm a staff engineer on the AI team here at Vercel and I've been on the AI team since its inception probably around two years ago. I was there for the AI SDK. I'm sure we'll talk about a little bit. And then I was there for V0 and before joining Vercel I was an intern there back in 2020, so been here two and a half years now and then, plus some change on the back end there.
- [02:06]
ARI Kandelwal
Nice. So you sort of bring us into our topic, the AI team, y'all, as I understand it, are both working on Shad CN as well. So do you want to kind of give us a little bit of overview, like what is this thing and what's the origin story?
- [02:21]
Max Slyder
Yeah, for sure. Real quick. So when it comes to ShadCN and the ShadCN components, we. We work very closely with them. ShadCN works at Vercel, but on the AI team we're largely working alongside ShadCN. So we want the components to not be AI specific. We want everyone to be able to Use them. You don't need to use Vercel or any of our products, but we work very closely with SHADCN to make sure that they work well for V0 in our use cases.
- [02:43]
ARI Kandelwal
And, you know, the AI team itself has two different products that we really work on. One which Max alluded to, is the AI SDK. The AISDK is a mini typescript framework that makes it really, really easy to build AI powered applications. The goal is, you know, everyone is building AI applications these days. The AI SDK lets you switch between models really easy. It gives you access to utilities that make building basic parts of this application, like streaming utilities and things like that, super easy. So you can really focus on the business logic of your application and not like how to stream from an OpenAI provider to. So that it renders in the UI super well, which is a solved problem. And the second thing that we work on is v0, and v0 is a tool for all developers to generate UIs on the fly. And yeah, v0 uses SHADCN components as a base component library to make those UI generations that it makes composable and reusable and actually built on top of a component library as opposed to being just spaghetti jsx.
- [03:43]
Got it. Okay, so let's maybe dive in first, then to that AI SDK a little bit. You mentioned a few different things. Would you compare this to something like LangChain or is there another comparable that people who haven't used it should use to kind of get their heads in the right mode?
- [03:58]
Max Slyder
I think it works with LangChain, for one. So we have a LangChain provider and we have some tools. So you can use LangChain with the AI SDK if that's what you want. And it provides a lot of more core primitives you could use to build something that LangChain might provide for you. Because I think, which is very common in software, is you use a library and then you hit some opinion of the library developer and you're a little stuck. So at the AI SDK, we try to be very unopinionated and a little more low level so you can piece together these parts and make your own pipelines, your own LLM apps, without us getting in the way of how you actually want to structure your program.
- [04:29]
ARI Kandelwal
Got it. Okay. And so looking at it, there's sort of a bunch of different pieces, right? There's like the core, there's the UI pieces, things like that. You know, if I were jumping into it and wanting to use the SDK, is there Like a mental model I should have in place for getting started with this.
- [04:45]
Yeah, I'd say there's two basic parts to AI SDK. One, as you mentioned, is Core. And core is really the basics around which the whole AI SDK is built. The easiest way to think about Core is like this is the way that you actually call an LLM. So instead of calling an OpenAI provider or an anthropic provider or whatever model you want to use under the hood, you call the LLM via the AI SDK Core, and you pass it a model which could be any model across the different providers, across anthropic and OpenAI and Llama and things like that. And the goal with Core really is just how can we make it really, really easy for you to switch between models, test different things out, and if a new state of the art model comes out, you should be able to use that out of the box without having to go back and rip out a provider and change a bunch of stuff. The second part of it is ui. And here we really focus on building reusable primitives that make parts of the UI very, very easy. Parts of the front end that are typically required for AI applications. Very, very easy. The basic component there is a function called Use Chat, which does a lot of the streaming behavior for you. It handles keeping track of messages and it handles rendering those as they stream in. So together they help you build AI applications pretty fast.
- [05:58]
Max Slyder
And I think kind of an encompassing idea at Vercel is we're always building for ourselves and we love dogfooding. So the AI SDK came out of us building the AI Playground, which is a web app we have on the aistk website that lets you try a bunch of different model providers all on the same screen. You have a bunch of columns, and then you're able to see the same response from like 20 different providers, and you can see how they all compare against each other. And we built this and we were like, wow, there's a lot of streaming code in here that's kind of tough to write. A lot of places you can mess up. A cool thing about Vercel is we have people here that know every part of the network or browser stack. So we had experts coming in and helping us fix our streaming code. And then we were like, why should we be the ones with this and why shouldn't we give it to everyone? So, so we pulled out the core from the AI Playground and made it the AI SDK. And since then, it's grown a bit, but I wanted to highlight on Ari's first point about Core, which is switching between models. Every model provider has their own APIs and they've kind of generalized around OpenAI's schema, but they all have different implementation details. They all have slight different things that are really annoying when you run into them. So the AI SDK covers all those. It gives you one interface to use all the different providers. And we also give you the ability to like when this model errors, switch to this model, do a lot more advanced things like that which are essential for production element applications.
- [07:14]
ARI Kandelwal
Yeah, so one of the things that is different across all these different models is that they expose different levels of completion stuff. So OpenAI, for example, seems to be standardizing towards chat completion and they've done a lot of training, so they're exposing that they've been deprecating a lot of times. Some of the lower level APIs, some of the models you might use are just like full on completion models. So I'm kind of curious, do you hide that layer? Do you have an abstraction above it? And what tools do you provide to let people dive through those layers as needed?
- [07:44]
Yeah, so I think the way we think about it is that like the base completion, like base inference is usually shared across all the different model providers. They all provide some kind of endpoint to do completions. In addition, they also provide additional like endpoints to do more specific tasks. And so the way that we've thought about it for the AISDK is that some of the functions that you can use in the AISDK are provider specific. So you still get access to the individual specific things that individual providers are giving you. For example, some models let you generate images, some of them don't. So the generate image functions in the AISDK can only be used with providers that provide you that image functionality. So our goal is that if providers come out with new cool endpoints that allow you to do new different things, we will still add those to the aistk. They might not be available across all the providers until each of them implement it. Typically what we've seen is that the rate of convergence for what these endpoints actually are is very, very fast. When OpenAI comes out with something new, Anthropic, Google, et cetera, all race to kind of provide similar functionality. And the same happens in the opposite direction if Anthropic or Llama or Google come out with something faster.
- [08:52]
Got it. Another question kind of related to this. So for example, OpenAI versus Anthropic is a good example. They deal with system prompts quite differently. Is that something that if I'm wanting to be able to swap quickly between model providers, you'll take care of for me? Like, how do you navigate that?
- [09:09]
Max Slyder
We don't manage the prompt format, although it's something we're exploring. Like, Anthropic is very strong about their models work really well with XML. OpenAI doesn't really say the same thing, but I think a lot of people use markdown for their prompts and we don't have a layer for doing that, but we do have the ability to tweak certain behaviors that they support differently. A really powerful thing you can do with models is called assistant response prefills. So you can, in your messages list, give an assistant message and then the LLM will pick up where that left off. So if you wanted to write JSON, you could put a little curly bracket there and that strongly hints at the LLM as you continue writing. JSON providers all treat those a little differently. And the AI SDK works around those bugs. So when you switch your provider or you switch your upstream, you don't hit those bugs. But overall, we try to keep it those changes to a minimum.
- [09:56]
ARI Kandelwal
That makes sense. Now, as we talk about building applications with LLMs, there's a lot more than just the like, model level interaction and there's some amounts of, I don't know if I'd call it standards, but like, patterns that are emerging around, like, how do you build agents and agentic behavior? How do you handle memory and context loading and things like that. Is that something that y'all are tackling as well or is that deferred to the end user?
- [10:20]
Yeah, 100%. So the way we think about it is that there's two ways we could have done this. One is we build more complicated abstractions and force you to use those abstractions for agent behavior, for doing rag, for doing basic things like this. We've purposely chosen not to do that. And the reason is because we've found that our users typically care a lot about having the lowest level primitives possible so that they can, you know, customize and do things at will. The way we really handled that is we provide you with recipes or like cookbooks on how to put together different primitives that we've provided you in the AISDK to do very common tasks. So I think like rag and agents and tool calling are like three examples of things that we have very good recipes for. So if you want, you can, you know, lift up our code snippets and just use them as is. But more often than not, what we're giving you this recipe for is that you have a good understanding of what was the mental model we had when we were building these primitives and how we thought about putting them together to do like, some of these more complicated use cases. And we found that this still allows you to have the flexibility to do whatever you want with the primitives, but still gives you an easy way to get started when you're like, hey, I want to do something complicated. I'm sure people have done RAG before. How do I do rag? With AI SDK, you can use our cookbooks and recipes for that. And one thing that's super useful for us is because, you know, V0 and AI SDK, like those teams work very, very closely with each other. At Vercell, a lot of the product influence for the AISDK comes from use cases in V0, because V0 is built on the AISDK. So it's very frequent that Max will be like, hey, AI SDK team, we're facing this problem with streaming, this kind of response on this provider. And the AI SDK can be very responsive and be like, oh, there's a bug there, we can fix it and update it. And a lot of the cookbooks and recipes we have are pulled out of patterns that we built into V0.
- [12:09]
Max Slyder
And real quick I'll hop in and say that a lot of what we release in the AISDK is also pulled from v0. So we build it for ourselves there and we're like, this is really good. We try to nail the abstraction and then we try to share it with everyone. One cool example I really hope more people start using is auto continue responses. So LLMs have an output limit and when they hit that, they stop outputting. There's no reason you can't auto loop back to the LLM and have it continue its last message. So now you can have really long code blocks or responses from LLMs with one line of code. And I think that's so powerful.
- [12:38]
ARI Kandelwal
So let's maybe, actually, now that you're referencing V0 a lot, dive into V0. So first, I guess from a user perspective, high level, what is V0? What does it let you do? And then I'd love to actually get into the patterns you're using inside of it to build it as an LLM application.
- [12:53]
Yeah, the way we think about V0 is it's, it's like an always on pair programmer that's an expert in all things front end. This Means that it's like an expert on topics like React, Next js, Tailwind, Shed, CN ui. All of this part of like the, you know, the modern front end stack. If you ask v0 very, very specific questions about new APIs released in next versions or you know, how to migrate to server components, how to use all these kind of new features in react and next, V0 is very, very good at talking about those. And the other part, which is what v0 is probably best known for is, you know, part of building front ends is building really beautiful aesthetic UIs. And V0 is, you know, most famous for being able to, given a prompt, generate UIs that give you the REACT code built with Next Full Stack applications that actually render that UI and you.
- [13:41]
Max Slyder
Can Preview them on v0 in the web very quickly without having to spin up a dev server doing these other things.
- [13:46]
ARI Kandelwal
Nice. So thinking about that, I'm going to dive into a few pieces there. So you said, okay. It's very good at answering particular programming questions. What layers are you influencing to do that? Like are you fine tuning some underlying model with more developer focused data? Or are you, do you have a search layer that you're then ragging that context into? Like, what does that look like under the hood?
- [14:08]
Max Slyder
So we can't go into too much details on like the specifics of the pipeline, but I think I can say that one reason we built the AI SDK to let you switch between models so easily is we don't want to be locked into a single provider or model. So so much of what we've done on the V0 engineering side, and what I think all good LLM apps do, is the non LLM engineering to make your results good. So that can be rag and having a great data set to rag against, or that can be having just a great pipeline of trimming down the user queries or doing all sorts of things. And I think that's where we've put a lot of effort. Like it can be very hard to make V0 write bad code sometimes. We do a lot of work to clean up the outputs. We're always improving that though. And I think it's really important that when you're able to switch models, you have to build around all their little mistakes and you have a really resilient product that way. And now if someone goes down or if we want to switch to our own for cell model someday, that's a very easy thing for us to just plug in and do.
- [15:03]
ARI Kandelwal
I think this gets into one of the interesting things in this Space, which is that, I think, led by OpenAI, the big model providers are often very fuzzy in their language about what's happening in the model layer and what's happening in applications. Right. We'll talk about, oh, people will say, I'm using ChatGPT. And you're like, okay, but some of that functionality is coming from the model and you'll talk about that. And some of that, it's making an API call off somewhere else, it's doing a tool call. Some of that it's, you know, ragging in the right data and it gets very fuzzy and people aren't breaking those down. So one, I love the idea of, like, separating that out and be like, all right, this has to be model agnostic. Some models might work better for this, some might not. But, like, the fundamentals of it are agnostic.
- [15:46]
Kevin Ball
In a world where agreements are the lifeline of businesses, DocuSign is more than just signatures. The company is transforming how professionals create, commit to, and manage agreements. Use DocuSign Intelligent Agreement Management to turn complex negotiations into a streamlined experience, breaking down barriers and pushing business forward. Visit DocuSign.com today to learn how DocuSign IAM can give your business a competitive edge and propel you into the future of agreement management.
- [16:16]
ARI Kandelwal
I'd love to dig into understanding. You can't go into the specifics of your pipeline, but how you think about what those different components are in building an effective application on top of an LLM.
- [16:27]
Max Slyder
Yeah, I think one good way to illustrate this is to think that when V0 started, it was August 2023. I think we started working on it a month or two before that. We had GPT 3.5 4K context. I think it did not know about SHADCN, did not know about new React features. So we were starting with a model that did not inherently know what we wanted it to do, and we had to teach it all of those things. And I think the key thing is having that data set and having a data set that you can rag against and then going really deep into the ragging. How do you chunk your content? How do you embed your content? Does it make sense to embed the user query? Does that user query really map to whatever your responses are, if they're code or images? So evals are really essential for doing that sort of thing. We've been on a wild ride figuring out how to do evals before anyone really talked about them, or at least publicly. So it's been a fun ride and I think always treating it like it's August 23rd works well for us. Like the model is always dumb. What can we do to make it smart?
- [17:20]
ARI Kandelwal
That is the fascinating thing I've found with these models is like, they seem smart at first chance and the more you use them, the more you're like, no, it's, it's dumb, it's dumb, but it can do some amazing things. So how do we make it smart?
- [17:31]
Max Slyder
Absolutely. And I think a big unblocker for us with how to make it smart was digging really deep into how users used it. Because when you give someone just like a prompt form or a text box, they can type whatever they want there. They can paste crazy things, they can be really rude to it, they can speak not English or you know, I, when I talk to V0, I often send like two words and I write out my whole sentence. Cause I know at this point what it will understand and take away from that. You have to learn those things first. And that just takes a lot of work and grinding and working with your model.
- [18:00]
ARI Kandelwal
I think the second part of that is like once you have a really good understanding of what your users think your application can do, you can make as many of those things non AI based as possible. So what we've tried a lot of is like, okay, the LLM is dumb. We know that it's dumb if there are simple things that we can do deterministically, especially if we have a very good idea of what users want. In those cases, we can pull those out, don't have the LLM actually generate those and do those deterministically ourselves. And we found that's a really, really good way to enhance output, which is like, let's not try to give the LLM infinite information and be able to do everything. Let's actually limit its scope, make it do something very, very well. And as much as we can pull out of that, that doesn't need to be done with the LLM. Let's not do it with the LLM. So like one example of this for V0 is like people very frequently want to change the text that they see in their UIs. There's two ways that this could happen. One is you put in some new text, we give it to the LLM and we're like, hey, LLM, go change this text with this new text. The problem is this is a non trivial problem for the LLM. It has to go find where you need to change. It has to do the exchange itself. And it takes a really long time. The user's waiting for an entire model. Call to happen and be returned to actually make this change happen. What we found is that the user actually knows where in the UI they want to actually change the text. We can use source maps to go back to the source code that actually renders this text and we can deterministically change the text there. And we can do that all without A making a model call, which means B there's very low latency for the user and C it's deterministic. It will work the same way every single time. The user can get a very good understanding of what's going to happen. Finally, it's more accurate. We do emails on all these kinds of things, but when we do a deterministic change, the chance that it's correct afterward is way higher than if we do a like non deterministic AI based chain.
- [19:48]
Yeah, no, I like that a lot. One of the things that I've been playing a lot with in my day job is we'll tell the LLM here's a domain specific language that you can use and then when it uses it, then we deterministically render the pieces of the DSL out into useful code or other things that might happen. And I can, I don't know if this is the case, but I can imagine with something like SHAD cn, you could expose to it the component model, but then just deterministically drop in the components as you need them.
- [20:14]
Max Slyder
Right, exactly. And I think something else we do, similar to what Ari was saying about the refinement is what we call it the selecting something and being able to type it without an LLM call is for example, LLMs are often lazy, so they will omit code. They don't want to write it all out. And we have a second pass that fixes the laziness. And I think that's a really powerful idea of don't spend a ton of time trying to prompt out these behaviors that are inherent to the models based on their training or whatever. Work with them. You know, if you wanted to output JSON and it's messing up the JSON, make or find a lazy JSON parser and fix that JSON. Don't try to keep retrying the LLM calls or wait for it to work. Make it work for yourself. And that's how, as I was saying before, like you build a really resilient product.
- [20:54]
ARI Kandelwal
Yeah. Another thing I've seen in this space that I'd be curious and maybe you can or can't say something about this, but tools like cursor not only have the core LLM call, but they have a diffing model that's like, okay, how do I apply this to the code that I already have? That's a separate model, totally fine tuned on its own and having different layers of Even if you are using models rather than deterministic output, but having different layers with more refined purposes can really up your reliability.
- [21:20]
Max Slyder
Absolutely. And that's a great case for like fine tuned models or small models or something you don't want for your main response, but you can use them to clean up that main response. And I think that's something super powerful that we'll be seeing a lot more of in this agentic 2025.
- [21:33]
ARI Kandelwal
Nice. So let's maybe talk a little bit as we're on V0 about SHADCN. You mentioned that's kind of the default component model. Can you explain a little bit what it is and why use that over something like Material as your default?
- [21:46]
Yeah, I mean shadcn is an open source component library built for React primarily, which gives you the most basic building blocks that almost every web application needs. So you can imagine the kinds of things you're getting here are like, like okay, cards, buttons, badges, dropdowns, modals, these are all like very, very basic parts of like any kind of web application that gets built these days. What ShadCN gives you is it gives you the code to render these components. And it's not an install, it's not a package or anything. You install it, it just gives you the code, you copy, paste that code and it gives you a really, really great starting point to customize and build your own design system. And so the biggest unique part about ChatCN is that you are not dependent on some third party system where material UI can update their package at any given point in time and change your design system. And you have to work around it. ShadCN gives you building blocks to start off with that you can customize at will and kind of build your design system from scratch around it. And yeah, that's ShadCN in a nutshell.
- [22:46]
Max Slyder
And I think it's also important to highlight shadcn components are built on Radix ui, which is a headless library, which means it has no styles, but it's very accessible. So you're starting off with Chad cn You're like set up for success. It's accessible, it looks good, it's very customizable and it turns out that it looks really good without a lot of code around it. So LLMs are actually pretty good at Throwing it together because they know there should be two buttons at the end of a form. It doesn't necessarily need to know what that button looks like to ensure it will look okay. It helps to give it some knowledge to make it look good. But it's a great place to start from where you're guaranteed kind of something that's reasonable.
- [23:18]
ARI Kandelwal
So you highlighted the thing that's different is it's not a third party dependency. It's essentially copy and paste, which is kind of an inversion of traditional best practice. What's the reason? Thinking behind that. And I, I think that's, it's ingenious, particularly in an age of LLM based coding. But I'm curious how you all think about it.
- [23:37]
Yeah, I think there's two reasons why this works here and it maybe doesn't work everywhere else. The first is like shadcn is not a static utility that once you use it once you want to use it exactly the same every single time you use it. So typically you want to import a package when it's like, hey, this is a date formatter, it will work exactly the same way every single time. And I want it to work exactly the same way every single time. If there are changes and updates on their end, I actually want to get those changes and updates. So that's the second piece. It's like typically it's a, I want it to work the same way every time and if there's a change, I actually want to get that change and be able to use the new version of it for a design system. Typically like both of those are not true. A, you don't actually want to use a component the exact same way every time. Components change and grow over time. It ends up being a block for you if you're like, hey, I actually need to modify this component slightly and suddenly I no longer have the ability to do so because I'm importing it from some package. I have to wrap it and use a wrapper for it instead and then change the wrapper. It ends up getting kind of complicated. And the second is you don't necessarily want all of the updates from like ShadCN. If he releases a new component, you can just use that new component, copy, paste it. But if you change the interface by which you interact with an existing component, you don't necessarily want to see that like get that upstream change. And so that's why it's actually, you know, our thesis is that it's actually better for this kind of design system use case to not have it Be a package that you import and like consistently keep updated.
- [25:01]
Max Slyder
Yeah, I think the key there is that you're able to change the components. So you really are. Once you install them, they're the default shadcn components. Then you can customize them as you're building your app or building your website. You start changing the border radiuses, you start changing the colors and all of a sudden it's yours. Now it's no longer really standard chat.
- [25:16]
ARI Kandelwal
Cm the approach here reminds me a lot of what you were talking about with regards to higher level abstractions in your AI SDK where you're not providing. Here's an installable thing with giving all these things you have to do, you're like, here's a building block. Go create.
- [25:32]
Max Slyder
Right. And I think that's all because we're building it for ourselves. Like this is what we would want and, and we would get upset if we couldn't change our button styles or if we couldn't modify how our rack works.
- [25:42]
Kevin Ball
The global developer talent shortage is expected to grow to 4 million in 2025, further contributing to developer burnout. With the security and talent shortage growing rapidly, businesses need effective tools to help developers work efficiently and securely. That's where Bitwarden comes in. Bitwarden delivers trusted open source security solutions that empower your developers and security teams to securely manage and share sensitive information online. Protect your infrastructure secrets, API keys, user passwords, mailing addresses, credit cards, passkeys and more. With easy to use and enterprise ready Bitwarden solutions. Start your free trial today@bitwarden.com.
- [26:22]
ARI Kandelwal
So bringing back a little bit to the AI SDK. We talked about core, but there was also SDK UI. Is that related to SHADCN? They're completely distinct. They play nicely. Like, what is the relationship there?
- [26:34]
Yeah, they're completely distinct. The way to think about UI is like it is utilities for you to use on the front end to do basic parts of AI applications. So a very good example is the most basic thing that people are building with AI is still chatbots, right? There's a lot of parts of a chatbot that are shared across every single chatbot that's ever been made. You have to have some kind of messages. The messages are either like, there's a role for what those messages are, they need to be rendered on the front end and they have to be like. Typically the AI messages get streamed in from some third party, like from some API. So those are shared across every single chat application, no matter what. The thesis with UI was like, hey, we don't really need you to come up with these abstractions from scratch. Every time you want to build an AI application, we know that you're going to build some kind of message object, we know you're going to render that message object, and we know you're going to need it to be streamed in. Why don't we give you like good abstractions for a message, for example, that you can use out of the box and that way you can get started with an AI application really, really fast. And the goal, again, the balance here is like we are giving you these abstractions and the goal is to keep them as low level as possible, such that you can customize them and use them basically at will, while still giving you the benefits of some abstraction where you're not worrying about the low level of how am I streaming in content and rendering it in the ui. Optimistically or not optimistically, UI kind of does that part for you.
- [27:59]
Got it. And for both of these libraries, you mentioned that the team works very closely with the V0 team. Are these going to be community projects at some point or is Vercel more or less run them top to bottom?
- [28:12]
I mean, AI SDK is an open source framework. Anybody can contribute to it. We have a lot of community contributors to it already, especially with when new providers come out and new models come out from new providers, you know, the community typically builds providers for the SDK for those. So AI SDK, totally open source already, very community driven. V0 right now is a like Vercel branded Vercel run product. It is built upon shadcn, which is itself another open source. You can open a PR to shadcn today if you wanted to. And that's kind of how Vercel thinks about all of its products. Even it's our managed, you know, paid products are built on top of very, very strong open source foundations. It's like built into Vercel's DNA.
- [28:52]
That's awesome. So looking now at these things, and you mentioned one source of changes, new models, things like that, you also said that, you know, as you uncover needs, often from VC or other things, you know, those get sort of baked down into the AI SDK. Where would you say kind of the growth edge is for these libraries? Like what are the pieces that aren't quite where you want them to be yet or where these libraries are really evolving?
- [29:21]
Max Slyder
I know something we're actively thinking about a lot. There's an RFC still, I think, on the GitHub, if anyone's interested in poking their head in There IS agents and OpenAI really swarm. And how do you coordinate multiple models together and then also render those outputs on the front end in a nice streaming friendly manner perhaps? So that's a lot of where our headspace is at right now. I think models have gotten so good you can now let them kind of do their own thing for a little bit. You can give their outputs to other models and we need some kind of tools to help pair those together a little bit more.
- [29:50]
ARI Kandelwal
Is that thinking advanced enough? You could sort of explain a little bit like, how is the AI SDK thinking about agents? Or how do you at V0 think about agents?
- [29:59]
Max Slyder
That is a loaded question because I know there's a lot of discourse right now about like, what is an agent?
- [30:03]
ARI Kandelwal
Hot take, what is an agent?
- [30:06]
Max Slyder
I have put a lot of thought into this. I still haven't convinced myself fully. I go more on the second. It's not just giving you an output. So when you are having it do tool calls to third party services or having it make decisions besides what to respond, I feel like that's agentic. You have multiple steps and it has to make decisions. And that's already fully supported, but now it comes into this multimodal agent where you have this whole graph of all these different things calling each other. And that's a really fun problem that I don't think I have a great answer to on if I consider those more agent or not. I guess.
- [30:37]
ARI Kandelwal
Yeah, I think people typically think of agents like there are two requirements. It must be able to do something, right? Like it must do something that is not just return text. And the second is it must be able to take multiple steps to do the thing that it wants to do. So typically, I would say if it can, if it satisfies both of those criteria, we call it an agent. There's obvious questions about, okay, if a single call can do multiple round trips of a tool call. Like, is that agentic behavior? Maybe Like, I think there's probably a. That's where the like discourse gets super granular and very in depth. But I think in general it's like, okay, if you're taking multiple steps and it is doing something that is not just returning a text response, we would call that an agent.
- [31:18]
So you described in V0 having, for example, loading different types of contexts, making decisions about doing things deterministically, doing other stuff, would you call V0 an agent?
- [31:31]
Max Slyder
Yes. Again, some people might argue about that, but it is definitely an agent.
- [31:36]
ARI Kandelwal
Yeah, I think the way people typically think about agents is that oh, if V0 generated a UI and then fixed it and then added a bunch of features one after another, all without any human intervention, that's what they imagine when they think agent. We already do a lot of steps for every generation that you get behind the hood that are hidden from the user. And v0 does a lot of things that are not just generate text response. I think we probably fit those two criteria. Although I know when people say they want an agentic V0, they typically mean like I want it to take multiple steps and generate a bunch of UI all at once. I mean it can take longer to do that, but yeah.
- [32:13]
So how do you think about how V0 plays into the product development process? And I know this is an active conversation all over the place of like, okay, what sets of things do you do with a V0 or replit or something like this where you're like generating something from scratch in kind of an agentic way and then when do you export it out and to handle it in a different way? I'm curious, what's your perspective? And since it sounds like you dogfood everything, how do you all use v0?
- [32:42]
So the easy answer to this is that we have found that people use V0 for everything internally at Vercel we have a channel on our Slack called howivzero, which is just people posting how they're using V0 for various different things. We have found that people use it for way crazier things than us on the product team ever would have imagined. In general, I think we've seen three to four big like large use cases. The first is early in the product development process when you have, when you're still at the ideation feature definition stage. Typically people spend a lot of time writing feature requirement docs or trying to build like low level mockups or some way get across whatever that product vision in their head is. V0 is a super easy way to get that across. And we've had multiple features at Vercel start as a V0 generation by some, you know, maybe non technical person and then actually make it to production because it was very clear what that, what that was intended to be. The second thing we've seen is people start with designs and they want to get like an 80% way there very, very fast. They don't want to spend three days like scaffolding out what this design is going to look like to build the front end for it. V0 is very, very good at getting 80% of the way there once you have a given Design and the third, the most v0 pilled way to use v0 is use it as a quasi IDE. Basically where you start the process there, you brainstorm, you work with V0 to define what this is going to look like. Then you build out the business logic. V0 can write full stack next JS code, you can build out the business logic in V0 itself. And then at some point when you're ready to deploy, V0 allows you to deploy to Vercel super, super easily. So you can really do the full product development process all within v0. And that's the like, yeah, most v0 pilled way to go about it. The final thing I'll quickly mention is that we've seen a lot of just like non app development use cases as well. So things like our marketing team, when they have to build visuals for data that they have, they give VZR the data and ask it to build really pretty charts. They can customize that, it's all backed by code. V0 can execute Python and execute in a Node JS environment. So people are actually creating scripts and running scripts, doing SQL queries, all of that kind of stuff, all in V0 as well. So there's a whole fourth category of non application development use cases that we found people use V0 for.
- [35:02]
Max Slyder
And I would just throw out that. I think a surprising amount of V0 has been built at least at the beginning with V0. It is such a good tool for I'm someone that I love writing code. I am not great at writing CSS or styling my websites. V0 is a fantastic tool for me. I do all the implementation myself or I'll have V0 start it. I give it the implementation and I'm like, make a ui. And it is fantastic for that. And I think for people that know how to code but don't know how to build front ends, it is an amazing tool right now.
- [35:29]
ARI Kandelwal
Let's maybe dive into a little bit of that back and forth. You said, okay, I give it my code and ask it to build a front end. What does that management of code process look like?
- [35:39]
Max Slyder
Sure. I'll preface this with we're actively exploring this. We would love to be able to have you plug in Git repos and sync it with Vercel and that is on the roadmap right now. I will take my schema file or my whatever file I'm working in, paste it right in. I might have a chat already where I made that file. So I just go back to that chat and be like all Right now I'm working on this part of it. I think Those are the two main ways. You start with V0 and as you use V0 more, you. You have chats about the things you're working on. So you can go back to those chats, you can fork your old block and you can have a lot more of your kind of like a git message, but it's, you know, a whole chat now. So you have all this context about how you got from point A to point B stored in this chat. I think that's super powerful.
- [36:19]
ARI Kandelwal
So if I'm hearing you correctly, right now, it's not directly connected to git. So if I wanted to go all the way V0 pilled and I wanted to build my whole application with this, but I'm, you know, a seasoned, perhaps burned a few times engineer and I want to make sure that I'm committing things along the way and things like that. Am I copying and pasting? Is there an export, if not an import? Like, what does that look like?
- [36:42]
Max Slyder
Right. You can copy and paste. Going back to shadcn, that's like we always want to support the copy and paste. We also have the SHADCN CLI, that's a tool that SHADCN wrote that lets you easily update and install your SHADCN components. You can also give it URLs to your chat or to your V0 block, which is the website in that, and it will pull all the dependencies, it will pull all the files, and it will make sure it's installed and integrated nicely in your project.
- [37:04]
ARI Kandelwal
Oh, so that's interesting. So I can have a chat with V0, end up with a component of some sort, and then use the shadcn CLI to pull that down into whatever repo I happen to be working in.
- [37:14]
Max Slyder
Exactly. And then you can edit that file if you want, or you can go back to V0, keep working on it and run the command again to update it or copy and paste. That's really interesting and it's cool that not super V0 related, but we're seeing more products around the web pick up this shadcn CLI support. It's open source. You can make your own registry. So one big one is framer motion or motion.dev now they have open and v0 buttons and like shadcn blocks, you can use the registry to pull Motion components into your React app. So that's one way again, we're taking this v0 feature and we're trying to share it with like, more of the open source community.
- [37:48]
ARI Kandelwal
So one of the things that I think we're all, all of us in software are grappling with right now is how these tools change what our job looks like. And y'all are not only building the product that's changing our jobs, you're using it to build itself and kind of doing that. So how do you think about what software development looks like in this LLM world we're in?
- [38:10]
Max Slyder
For me, it really is you can just build more instead of you working on one thing at a time. Now kick off 3v0 chats, I'll tab between them and I'll be working on three things at once while another page is responding. And I think as we go down an agent route and like all these models, start doing more multi step things, you'll have more time to wait while your output is being done, which gives you more time to do more things. So maybe it's naive of me. I really am thinking it's like a force multiplier and that's how I've seen it help me. I just am able to produce more code. I might have to review it a little more carefully if V0 wrote a lot of it, but that's still faster than me sitting there and writing it all by hand.
- [38:44]
ARI Kandelwal
I think it also lets you work at a higher level of abstraction, especially when you're working with syntax that you're not super, super familiar with. So when you're working with new APIs, like new functions, new languages, anything like that, AI makes it really easy for you, as someone who understands computer science and principles behind development, to build things really, really fast without having to spend the time going through and making sure your syntax is accurate and looking through documentation to figure out what functions call and stuff like that. So very frequently What I'll have V0 do is I will give V0 a tech plan of sorts where it's like, hey, I need to load this data in a server component, pass it down to a client and write these hooks, which is in my head how I plan out what I have to build. And I'll just say, okay, you go build that. And so the architecture decisions are still being made by me, but the implementations I don't have to worry about because those are like relatively straightforward given the architecture decisions.
- [39:41]
Max Slyder
And it certainly helps if you're technical, so maybe, you know, you can guide it a little bit on debugging or what to use. We see a lot of people using V0 that are not technical and we just try to make it really Easy for them. Like, you might get an error screen if v0 writes wrong code or you visit a page that doesn't exist. We give you a nice little button that says fix with v0 and you hit that and now your error message is sent to V0. So trying to streamline the development process also makes it easier for us. So we don't need to copy and paste the error message. And people that don't know what an error message is. And I think that's been really powerful for the adoption of V0 and then for all sorts of people at companies be able to use it, not just the engineering or product teams.
- [40:16]
ARI Kandelwal
In fact, the top user of v0@vercel is not an engineer. Actually, I think the top three people who use V0 at Vercel are all not engineers.
- [40:25]
That's kind of interesting. What would you. And you have this whole internal channel of made with V0 or whatever. Like, what are some of the most novel use cases you've seen for this?
- [40:35]
Some interesting ones that are like, maybe not super novel, but like, were not things we were thinking of is our sales engineering team will use V0 to build customized demos for prospects. So instead of coming with like a slide deck, they will come with a demo of like whatever it is that they're trying to explain to the prospect, which works way, way better. And what they can do then is live during a customer call. They can modify the demo based on things that the prospect is saying. So this is never not something we had thought about at all when we had built V0 in the first place. But we've seen tremendous usage among our sales engineering team to make that happen.
- [41:12]
Max Slyder
I think one of the coolest demos I've seen was, I think Guillermore CEO was giving a presentation.
- [41:17]
ARI Kandelwal
Oh yeah, some talk.
- [41:19]
Max Slyder
And he gave the presentation using Google Slides and everything. And at the end he had escaped. And it was Google Slides built entirely in V0. The UI matched. It was started with a screenshot. You could add slides, edit your slides and hit present all in this V0 generation. I love talks that do that. I thought it was really cool. And it actually we keep using that. We keep forking the block and changing it for different things because it's such a fun way to build a presentation.
- [41:40]
ARI Kandelwal
No, that's really interesting. And I think one of the things that these tools are enabling is that much more rapid iteration, right? Of like, okay, I'm building a demo, then I'm talking to them, then I'm changing it and it's evolving and that Lets you really explore much more quickly. I am curious on the productionization side. And this is the classic engineering concern of like, okay, it's great to build a demo super fast. And the CEO says, all right, that's great, we're shipping it next week. Right. What does the productionization path look like? You alluded to this a little bit, but like how much work does there tend to need to be to get it from the 80% demo able version to the 100% I can sell this to customers version.
- [42:23]
Max Slyder
I think it depends on a few things. One is your experience and your ability to code. We have people that have 200, 300 versions of their block in V0 or their generation and those will often be, you know, 20, 30 files. It'll have Suvabase, it'll have login and auth. And those are effectively production ready. People hit deploy and they deploy them to Vercel. It took a lot of prompting for them to get there, but they got there. If you're trying to beat a deadline and you're an engineer, you might want to pull it out of V0, put in your code base and start working yourself. And those are both fine and we've seen both happen. I think what's key is that like by giving V0 the up to date knowledge on how to use next JS, how to use React web standard best practices, all this knowledge we have at Vercel that we've given V0 to try to make it the best web development agent or LLM, that all pays off in dividends. When you actually go to deploy your application and like, hey, it added rate limiting to the server route or it properly didn't block on this really long request or something like that.
- [43:21]
ARI Kandelwal
Yeah, it is interesting. I find particularly like some of the models from Anthropic like Sonnet, they write better code than I do when I prompt them correctly. Right. And like if you can code some of that prompting knowledge into your application layer of V0, like, dang, you get good stuff out.
- [43:36]
Max Slyder
Yeah, absolutely. I think one thing we had on the original V0, which we didn't talk on, but that was not a chat app. You gave a prompt, you got Generations, no chat. We gave you three versions of something and that's because models were a lot worse then. So we kind of had to try it three times and hope that one of them worked really well compared to the other two. And I don't know, I think still there's something there about giving people the choices and the ability to kind of pick and choose well.
- [43:59]
ARI Kandelwal
And I think that is one thing that's key in all of this. Use of LLMs for stuff is like you can't turn your brain off, right? Like you need to be engaged, you need to be checking. Is this actually doing what I wanted? You know, you need to look at it and approve it or correct it. But yeah, it makes that in between process of. I have an idea to. Let me see a first version of it just goes so fast.
- [44:22]
I would say the analogy I often use here is that like it's like a higher level programming language. Like it doesn't mean you need to stop programming. You know, when you stopped writing, you know, assembly and you started writing C, it didn't mean that like you stopped programming suddenly. It meant that you were still just using programming concepts at a higher level of abstraction. You still have to be able to do that in some capacity here. And this is just another higher level of abstraction that you're now able to work at.
- [44:47]
Now you all said you started this project, if I recall, it was like summer 2023, something like that. So we're now a year and a half along. If you project forward six months, a year, a year and a half, what do you see happening in this space in General and with V0 in particular?
- [45:07]
Yeah, I think we can maybe talk about what's happened since we started and then what happened in the future. So as Max mentioned, When we started V0 in October of 2023, it was like OpenAI's 3.5 model class and 4K context window, which now seems like the craziest set of assumptions to begin with. So a few things have changed. One, models have gotten like 10x smarter and we expect that that will continue. So the cool thing about building applications on as a layer on top of like base LLMs is that when LLMs get better, the applications we built get better. The ISTK makes it super easy for us to keep using the state of like state of the art model as it comes out. So the number one thing we expect is models will get better, our code quality output will get better, and LLMs will get better at coding. The second thing is that context windows are increasing. So at first you could only give the LLM a little bit of information alongside your prompt to make it do whatever it is that you wanted it to do. You can now give it probably 10x that amount of information. And in fact, Gemini has released models that have an infinite context window. You can give it as much information as you want. So the ability to get Custom personalized answers is getting better. Obviously there are infinite context windows are not as good at dealing with all that context as shorter context windows. There's more abstractions and more work to be done there. But in general you're getting more personal, better responses. And the third is that we're getting better at just building the software 1.0 layer here of building good web applications around AI. We're getting better and better at, at being able to do that. So things like being able to do some things deterministically, being able to use different models for different pieces of our generations and things like that, those are all only going to get better.
- [46:57]
Max Slyder
And I think it's really clear now that all the major model providers, they're aware of the V0 style or anthropic artifact style use case of generating code, probably React and chatcn and Tailwind. So now we're expecting models to get even better at maybe that subset of technologies because it's so popular right now and it's so good for us as developers.
- [47:20]
ARI Kandelwal
So I'm going to push a little bit more because yes, we can expect the models to get better. What do you see missing right now in v0, like what would you want to build or what are you already building that's going to make this.
- [47:33]
I would say, like one interesting case is, let's talk about error rates, right? So given an LLM output, there is, let's say an x percent chance that like that output has some kind of error in it. Right now our problem is when we want to do more complicated tasks, we can't stack those on top of each other because we compound error rates over time. So if we have a 10% error rate on one generation and we have four to five different generations happening, you now have one minus x percent to the nth power chance of that being wrong, which compounds very, very fast right now with the error rates that we see in models. Even three to five step process is not something you can trust the LLM to do because even in just five steps you're at a 50% success rate. So just a tiny increase in the success rate of base models allow us to stack model outputs and feed in the output of one LLM into another LLM in a much more agentic fashion and just do more complicated things as a result. And small increases in accuracy dramatically affect our ability to do that because of the fact that these errors compound over time. So I think that's one example of at least something I'm super excited about is that as Models get better, even slightly incrementally, we can suddenly stack a lot more of those together without seeing a huge massive increase in our error rates.
- [48:52]
Max Slyder
And I think we'll also see LLMs get better at using just modern technology. They're not necessarily trained on a big thing documentation sites now are doing is they're exposing LLMs Txt so you can visit this. Txt file and it has all of the docs in one file, so you can copy and paste those to an LLM or run your rag and chunking on them. And I think tricks like that are becoming so much more popular. Anthropic has MCP which lets models connect to basically integrations. So V0 lets you use environment variables from your Vercel projects. You can connect to anything your production website can. I think we'll see a lot more of these integrations and third party experiences kind of merging. So why can't you render part of your supabase dashboard and v0? I think you will be able to soon.
- [49:35]
ARI Kandelwal
And I think part of it is like we've been talking about changes that model companies and at the AI layer, what's going to change to get better. But the world is changing to adapt to how people use AI. And I think the LLMs txt is a really, really good example of documentation sites are changing how they do things to be better ingested by LLMs. One of the reasons, for example shadcn works so well with LLMs is that the styling and the component structure are co located. You have inline styles like tailwind class names, styles where the structure of the component and the styles for the component are in the same place. And one of the reasons why this is so good for LLMs is that they no longer have to keep track of two different files and try to keep them together. So more people are adopting Chad CN and more people are adopting React. More people are adopting tailwind because LLMs are good at this kind of thing. So basically the world is also changing to reflect what AI is very good at.
- [50:31]
Yeah, absolutely. We're learning how to use these incredibly powerful tools and we're building all that those connecting tissue. I love the idea of being able to pull in your dashboard from Supabase or whatever, run it in your V0. We were talking about import my git project, make these changes by chat, export it, deploy it, do all these different things. That's an exciting world. So we're getting close to the end of our time together. Are there any things we haven't talked about that you all want to make sure that people take away knowing about v0, the AI, SDKs, SHADCN, any of these different pieces.
- [51:08]
Max Slyder
I think I would say that we are building faster and faster all the time, largely thanks to V0 and AI. So I would expect that to be true for a lot of LLM applications. But they're only going to get so much better from here at a much quicker rate. We're all figuring out what works, what doesn't. People are writing about it and sharing. And I think that V0 and AI applications, if you've used them like a year ago, revisit them, try them again. Everything is just so much better now.
- [51:35]
ARI Kandelwal
You can get started using all of them for free. You know, you can just go to v0.dev and try out some prompts like play around with it, see what comes out. The AISDK is totally open source now. You go to SDK, Vercel AI, you can see the docs and start building applications with it right away. And Vercel has a lot of great templates to actually get started with all these kinds of technologies. V0 also has templates to get started with building really cool designs. So basically there's no excuse to not.
- [52:03]
Be coding, regardless of if you're a coder.
- [52:07]
Max Slyder
Yeah, exactly.
- [52:08]
ARI Kandelwal
Everybody can ship, everybody can cook.
- [52:10]
I love it. That's a great place to stop.