← Software Engineering Daily
Software Engineering Daily
Docusign for Developers with Dan Selman and Larry Jin
00:45:10
Agreements and contracts are a fundamental innovation and govern everything from personal commitments to major financial decisions. They function as trusted artifacts to capture the nature of a commitment and provide clarity and accountability.
Loading summary
Transcript57 lines
- [00:01]
Shawn Falconer
Agreements and contracts are a fundamental innovation and govern everything from personal commitments to major financial decisions. They function as trusted artifacts to capture the nature of a commitment and provide clarity and accountability. Software has revolutionized many business functions, including the basic mechanics of digitally signing an agreement. However, the process of managing agreements systematically and at scale, with type definitions, programmatic document creation, and storage schemas, remains a complex and largely unsolved challenge. Dan Selman is the product architect at DocuSign and was previously co founder and CTO of the Smart Agreements platform clause. Larry Ginn is the VP of Product Management at DocuSign and previously worked at Amazon, Microsoft, and Google. DocuSign recently released a developer API focused on fully modernizing and scaling the agreements process. In this episode, Dan and Larry joined Shawn Falconer to talk about the frontier of digital agreements. This episode is hosted by Shawn Falconer. Check the show notes for more information on Shawn's work and where to find him.
- [01:20]
Dan, Larry, welcome to the show.
- [01:22]
Dan Selman
Hi.
- [01:23]
Larry Ginn
Thanks for having us.
- [01:25]
Shawn Falconer
Yeah, absolutely. Thanks for being here. So we're talking about digital contracts and agreements today, but before we get into all that, since there's two of you, so let's have you introduce yourself. So, Dan, let's start with you. Who are you and what do you do?
- [01:37]
Dan Selman
Yeah, thanks, Sean. Thanks for having me. My name is Dan Solman. I'm a product architect and distinguished engineer at DocuSign. So I've kind of worked my way up through the tech stack. You know, I've done most of the engineering jobs as an individual contributor, and now I do a lot more architecture and strategic work.
- [01:55]
Shawn Falconer
Awesome. And Larry, same question to you. Who are you and what do you do?
- [01:58]
Larry Ginn
Larry Jin, I'm VP of Product Management here at DocuSign. I lead a number of areas, but really focused on building a platform for our developers and partners to build and extend our agreement capabilities. Before this, I built platform products as well at Alexa, at Amazon, and Microsoft as well.
- [02:17]
Shawn Falconer
Awesome. What ended up, you know, I guess, like, interesting you in DocuSign and what. What's particularly interesting about the space that led you here?
- [02:26]
Larry Ginn
Yeah, first off, well, what personally led me to DocuSign was the brand, the familiarity with the brand. I had good friends who worked at the company and said nothing but great things about the people and the culture. But what's really kept me at the company, I would say, is I think it's a pretty interesting space. Agreements and contracts, it's something that everyone deals with at all times. I'm sure everyone signed an agreement or a contract for whether you bought a house or rented an apartment, bought a car, even things that you don't necessarily think about day to day. Giving consent to share your health care record information with one of your providers, if you're going to the dentist, if you're going to the hospital, hospital. But it's kind of interesting how little sort of innovation there's been historically on this front and how much opportunity there is just given. People deal with it every day. Businesses deal with agreements when working with each other and contracting with each other. Individuals, like some of the examples that I just mentioned, you're dealing with agreements all the time. But a lot of the processes, even though DocuSign has spent a lot of time digitizing kind of that signing experience and making it easy to sign that document, there's still a lot of parts of the workflow around it that are pretty broken, disconnected. You know, we could talk a lot more about it, but that's really kind of the heart of what DocuSign is doing right now. And I think what's pretty interesting in this space.
- [03:45]
Shawn Falconer
Yeah, I mean, it is kind of, you know, when you take a step back and you think about like your interactions with business or business to business, things like you said, renting an apartment or buying a house or whatever it is, there's so much contract agreements, like paperwork involved in that process and it's, you know, stayed essentially relatively the same throughout, like decades and decades of essentially the same paper processes.
- [04:08]
Larry Ginn
Yeah, absolutely. There's an analogy that I think works really well here. You think about maps, you know, you think about the journey that kind of maps have gone through. Twenty years ago, if you wanted to go off on a road trip somewhere, right, you'd probably have a physical version of the maps, that giant thing that you would unfold eight times, and it, you know, be the size of like, you know, take up your entire dashboard. And that's how you kind of navigated and got around. And then, you know, we transitioned to digital versions of maps where you could kind of have a digital representation with MapQuest and things like that, but, you know, they would still be kind of clunky because you would still end up printing it and then having a version of it printed offline with all the directions and turn by turn, you know, then what happened? Then you got, you know, mobile phones and you had the ability to use GPS and pull up the map in real time and directions. And now you think about the experience. It's so integrated, it's so seamless. It integrates with your apps. You know, you look at the restaurant that you want to get to and it automatically kind of takes the directions. You know, now with cars, it'll sync it to your heads up display. And we think about kind of agreements as we've gotten, you know, one big step forward in the last decade or so with taking what historically was this piece of giant stack of papers that you would have to read through, you'd have to initial 17 different places, you'd have to sign. And now we've got to a point where that representation is digital and you could sign a PDF version of it, which is obviously a lot better, but there's still so much more left to do. I mean, all the steps leading up to creating that agreement, you know, making sure that you have the right templates, the right language, the right clauses. I mean, maybe some of our listeners probably don't have too much experience with having to go create this stuff. It's actually a lot of work and it's super complicated. You know, the exact language and specifics of what clauses and terms you can use or not use. And so there's a lot that actually goes into that creation part of it. And then there's the actual signing of it, which, you know, seems simple again, but it's actually pretty complicated depending on where you are. If you're in Europe, there's some pretty heavy requirements around proving who you are. You have to make sure that, you know, you upload a copy of your passport so that you have your identity proven. You might have to notarize it. You know, in some parts of the us, depending on the transaction, there's a lot going into signing it. And then the really interesting part, Sean, is what happens after that gets completed, right? What happens after you sign that agreement? So, you know, most of the time you get a PDF, you have the signature stamped onto it and then, you know, it kind of goes into a box somewhere. Right. But in reality there's a lot more that you have to do with it. You might have to take all that information and you have to throw it into your database. You know, think about like a banking transaction. If you're going to create an account, if you're going to update, you know, your personal information, your address, all that information that comes off of that agreement has to go back into that system, into that banking system. Same thing with those business to business transactions. If one company is buying something, you're buying widgets from another company. The information about how many widgets or the price per widget, when is it going to get delivered, for how Long, all that information has to go back somewhere and then companies have to go retrieve that information. They have to go find out what got signed, how many units did we buy, what kind of discounts did we get. And depending on what industry you're in, that information also becomes really specific to your industry. If you're in manufacturing, you know, you care about the quality, the tolerance of the manufacturing, things like that. So that is kind of the entirety of the space. And I think what's kind of interesting about DocuSign right now is, you know, we were really good at that first part, which is helping you sign that PDF, right? Getting it completed, making that experience a lot simpler so you don't have to literally download a stack of papers, print it out and then, you know, sign it with a pen. But again, I think there's so much more to do. And what we've, you know, launched is kind of this new product, this new offering called Intelligent Agreement Management. We launched it last year and it's really to help in all aspects of these processes for kind of all different kinds of customers. You know, we have small customers like mom and pop shops, all the way up to the really, really big ones. So that's why, you know, I, I'm interested in this space. And it continues to be a pretty interesting one that I think, you know, a lot of companies don't realize that there's such a big opportunity here. You know, I won't bore you guys with all the big numbers and like the market stuff because then I'll catch flack for being the product guy. But, you know, it's a pretty big opportunity. I think we're both excited about it.
- [08:21]
Shawn Falconer
Yeah, it's interesting. There's I think this like well worn path with these transformations that happen when you're going from like sort of paper to digital. The first sort of step of that transformation is like let's literally take the paper process and make like a digital version of that. And that is like a step forward, but it's not like an innovation step. It's like we've gone from physical maps to kind of like a map manifestation on the Internet or something like that. And then now we have a source like experience with maps that's hard to even imagine interacting with maps in a different way. And it sounds like sort of the E signature part of that was maybe the first step of making this paper process a digital process. But then there's so much more that you can actually start to innovate with. You know, Dan, there's a lot of essentially money, in my understanding, that's lost essentially like poor agreement management. Like you mentioned, Larry, the intelligent agreement management platform, like how does that fit into trying to help businesses essentially reclaim some of that value that are lost due to poor management?
- [09:20]
Dan Selman
Yeah, that's a huge motivator for most of our customers. You know, as Larry said, agreement management is that sort of tricky last mile for most business processes. So over the last few decades, we've digitized most business processes in the back office. But that last mile, like getting the signature, getting the agreement done, has been surprisingly resistant to digitization. And that leads to these inefficiencies. So I think there's something like $2 trillion or something is lost due to inefficient contract management. And that comes in a couple of forms. So on the one hand, you may be signing things that contain risks that you're not aware of. So you can think about things like service level agreements. Perhaps you're signing up to very aggressive service level commitments with your customers. And perhaps you're not aware exactly of what those levels are because as Larry said, they're kind of just baked into PDFs and, you know, stored on a file share somewhere. And it gets very difficult for the human beings to manage all those commitments. On the other hand, you may be entitled to something that you're not aware of. You know, maybe your supplier isn't meeting their service levels and you could be getting discounts or better business terms that you're not aware of. So that's in aggregate every day, millions, if not billions of agreements assigned, they encapsulate these rights and obligations. And when we don't manage those rights and obligations efficiently, then we get this contract leakage, this agreement trap problem, as we call it at DocuSign.
- [11:03]
Larry Ginn
Yeah, I'd love to maybe add another couple of examples. One that comes up a lot too. Like the one that Dan mentioned is renewals. We call it sneaky renewals. And you get these contracts that get signed, you know, for could be a vendor, could be some software license in your organization for 12 months, right. And then it says, oh, you know, this contract is going to auto renew on January 1st the following year. And then everyone says, okay, well, you know, we'll think about it one year from now. Well, we'll make sure somebody has a reminder in their calendar to go, you know, renegotiate this or maybe revisit, like, do we actually need this particular tool? Or, you know, software. And then, you know, guess what happens? Like, no one does that. Right. And then Gets auto renewed, and then you're on the hook for another year of spend, whether you like it or not. The other one that comes up a lot too is actually having to take the information out of those agreements and then put it back into your main database. I talk to a lot of our customers in like banking, like credit unions, you know, these are smaller organizations that still move a lot of money around, but they're not necessarily, you know, the most technology savvy. They're not, you know, leading IT firms. And so they end up having people back office folks who spend a lot of their time kind of just taking that information we call the swivel chair. You know, you get information from one place, you swivel to the other part of your screen, another tool, and then you just have to manually reenter that information. Right? And it's like, well, you know, we got you a digital version of the agreement, but it's really not that much better than taking a physical version and retyping everything into your computer. I mean, it's only kind of a half step better than that. So, you know, there's a lot of opportunities to kind of just make those everyday life simpler for folks like that.
- [12:38]
Shawn Falconer
Yeah, I mean, you're taking a lot of. Essentially all these agreements and contracts are unstructured data. It's just a written contract, but they encapsulate a lot of these rules, SLAs, when renewals happen, all these types of things that it's very easy for that to pile up and slip through the cracks. And I think historically, businesses essentially just haven't had a way to mine that information other than assigning a person to go and essentially transcribe that into a second.
- [13:03]
Larry Ginn
Exactly.
- [13:04]
Shawn Falconer
So how are you solving that problem? Are you essentially bringing in some of the latest developments in generative AI to help you mine and automatically extract some of that information?
- [13:14]
Dan Selman
Yeah, for sure. So as Larry said, we kind of think about this as an agreement process, right. You go from wanting to enter into an agreement, so you probably produce some sort of draft or template, negotiating that agreement, signing the agreement, and then post signature, you have to manage the agreement. And AI applies to all of those different phases. The one you touched on was probably like, okay, we've signed this unstructured thing. How do we get some structure out of it? And there obviously, you know, all the latest cool stuff in terms of NLP and LLMs is super applicable there. So we have a range of data extraction models that we can apply to unstructured documents and they pull out those Semantic elements of agreements. So not just like the names of the parties and the expiration date or something, or the renewal date, but actually the fact that, okay, this agreement contains a renewal clause, is an auto renewing agreement, it will auto renew on this date. And we build this very rich semantic model of an agreement based on dozens and dozens of data extractions that we then index and make searchable and expose via our Navigator user interface.
- [14:26]
Shawn Falconer
Can you walk me through a little bit about essentially the life of a contract and what's happening on the DocuSign side? Like what parts are coming into play in terms of being able to improve the process, but also automate some of this.
- [14:38]
Larry Ginn
So there's sort of these, you know, three big parts of that journey. You know, the life of an agreement, we call it. Right. So the first is the creation of it. And like I said, there's actually a lot of kind of different steps that go into it. Then there's, you know, the actual execution or committing where both parties come together and actually sign the thing. And then there's all the stuff that happens after what you call management agreement management. So the creation part, you know, really starts with what is this agreement that we're trying to get signed by counterparties? And that usually starts with a template. So a template could have been prepared by, you know, the legal team or, you know, a bunch of people coming together. You author something in Word or Google Docs and that forms the template and the basis of the agreement, you know, could be like MSA or sales agreement. Could also be like a apartment lease. Right. You know, usually it's a template. People don't create those every single time you want to get something signed. So once you have the template, then, you know, you need to fill it with kind of the information specific to the person that's signing it. Right. And you think about anyone that's ever signed an offer letter, you can kind of tell which parts are boilerplate and which parts are specific to them in terms of, you know, their name, their title, their start date, maybe their offer package, things like that. So that's sort of the like dynamic information that gets bound in, you know, from whatever system is generating that contract. You know, it could be like Salesforce or it could be like a workday, things like that. Sometimes there's also, you know, you might have to dynamically pick and choose different language and parts. So, you know, there's oftentimes what state you're in, what country you're in, there's additional regulations, you know, like people who are Signing employment contracts for a company that spans lots of different countries. If you're based in the UK or based in France, you know, you might have very specific language and additional, like, documents, additional writers that kind of say additional terms, right? So all of that kind of gets thrown together into this thing we call document generation. And then, you know, depending on whether or not there's additional approvals, it might go to your legal department. So there might be reviewing. And, you know, what we call redlining, which is kind of looking at specific terms. And then literally, you know, lawyers used to take a red pen and used to underline kind of the terminology that needs to get changed. Because they say, well, you know, we don't like this, we don't like that. So that draft document then, you know, will get sent over to the counterparty, and that goes to the second part, which is the actual execution of that agreement, that. The signing of it, right? And so when two businesses get together to sign something, agree, you know, to buy something, they'll send the paperwork over, and then the legal team on the other side will review it, maybe make some edits, make some revisions, and then eventually get signed. You know, if you're buying a house or buying a lease, you know, you probably won't have your lawyers review it, although sometimes you do. But in most cases, you kind of just sign it. You might have to notarize it, depending on, you know, if you're like buying a house, you might need an online notary. And then eventually, you know, once it gets signed again, that saved version is kind of set in stone, you know, memorialization of what was agreed to those terms that, you know, Dan was talking about. And then eventually, depending on what was in there down the line, a year later, you know, two years later, you might need to go back to that agreement. Like, no, this happens a lot where all the time you get, like, some new regulation comes out, right? Like GDPR VNext comes out. And then companies are like, oh, man, is there anything in our contracts that this might actually be problematic? We might have to go back and change things, or we have to reach out to our clients and customers that sign signed a version of our agreement years ago, and we might have to change something. And, you know, that happens a lot now where the regulation is moving so quickly. You know, new things get introduced, things get rolled back, right? And so legal teams are always kind of scrambling to figure out what was agreed to before. And then what do we have to go change or who do we have to reach out to to go make sure that we're in compliance. So, you know, it's pretty tricky stuff and like you kind of look at every single part of it and this is opportunity because we've done a really good job with that one step in the middle. But there's a lot of others that are still pretty manual today.
- [18:27]
Shawn Falconer
So where are you from, like engineering perspective, like applying AI? Other forms of automation help essentially at each part of that. So you mentioned sort of being able to extract some of those rules automatically through understanding what's going on in the document once you've sort of in the last mile of the contract. But what's happening ahead of that? Are there other places where you're applying some of that type of technology?
- [18:48]
Dan Selman
Yeah, definitely. First of all, my background is in good old fashioned AI, which it's all about rules and process and expert systems. That's how I got into this space. If we zoom out, what our customers are trying to do is to digitize these agreement processes. Most of those agreement processes are deterministic and precisely described. We need to do this, generate the agreement and then we need to send it to the counterparty for negotiation, and then we need to sign, etc. So the first thing that we have is a process designer essentially, which we call maestro, that allows the process creators, wiring together boxes and arrows in a fairly classical way, to describe those agreement processes. Then what we see is within those agreement processes there are specific steps where it's useful to transition from unstructured data into structured data. So in the front of that process, it could be during negotiation. So Larry and I are negotiating an agreement. He wants this term, I want that term. First of all, what's the difference between the terms? Can we have AI summarize the red line, essentially? So that will be a great accelerator for the people that are involved in negotiation. They're perhaps operating at slightly higher level abstraction. I will see a summary of Larry's changes. Larry's requesting a discount of 70% instead of 65%. And then we get into approval, right? It's like, well, do I have permission to approve that? Or does it have to go to, you know, a VP or GM or something to approve that level of discount? So that sort of front end application of AI summarization differences. And then there's also the description of some rules. Usually these negotiations, they're not kind of free form, right? When we go into negotiation, we have a set of rules that often is called a playbook. So when Larry does X, typically what do we do? Okay, if Larry asks for a discount above 50%. Well, typically we will, you know, offer up to, you know, 70 and perhaps change some other terms, etc. So understanding those rules, interpreting them as natural language and then applying them during negotiation is again, it's is like a key application of AI. You know, once with post signature we've signed this unstructured document, in most cases it may have been negotiated, it might have deviated away from our templates and we need to get back to some structured information that we can reliably index and search. And there we do, you know, fairly classical NLP data extraction. We use a variety of models, some built in house, some open source, some, some commercial. And as I say, we have dozens and dozens of these models that we apply to these incoming documents. And probably the hard part of that from an engineering perspective is once we've extracted the data, we need to understand the semantics of the data and we need to land it into a semantic model of an agreement. So we've built out a very rich definition of an agreement. An agreement is composed of clauses. A clause could have a type, one of the clauses could be a renewal. Renewals typically renew on a certain date or auto renew and so on and so forth. I call it like peeling this semantic onion of the agreement. We'll drill down deeper and deeper into the semantics of different types of agreements with the ultimate goal being that you can get them into Navigator where you can slice and dice them and run reports on them and say, all right, find me all employee agreements signed in the last three months where we offered, I don't know, more than half a million RSUs and the employee lives in California and they have a non compete agreement more than 24 months. That's the kind of information that we want to give, but at a macro scale, right? So if you're a global 2000 company signing thousands of agreements a day, you get this cockpit overall dashboard that gives you that visibility.
- [22:53]
Shawn Falconer
In terms of building semantic model for these agreements. Are you starting with some sort of base model or are you essentially, bottoms up, sort of reverse engineering that semantic model based on the things that you're able to pull out the entities from NLP and then essentially figuring out what those relationships are and then continuing to build sort of this ontological structure that describes the agreement.
- [23:14]
Dan Selman
So as part of the product, we give you a base semantic model that we've developed over the years. That's pretty rich. And then that model has some extensibility mechanisms built into it so that I think we ship around 30, 40 standard agreement types, so things like employee agreements, offer letters, MSAs, sales agreements. But we kind of understand that customers do all kinds of unusual things and they might need to extend that set. So the ontology is designed to be extended for are given either vertical, which our partners will be able to do, or into a customer account. Customers can directly customize the ontology.
- [23:57]
Shawn Falconer
How do you test all this? You're using a lot of different models. You mentioned summarizations. You're probably using some generative models, but you're also using sort of classical predictive ML along the way as well. How do you sort of combine all those things and make sure that the output essentially is matching the customers and your own expectations and that you're continuing to prove those things?
- [24:17]
Dan Selman
Yeah, I think that's a great kind of segue or point. Right. It's like it's not enough to just have an OpenAI account and upload a PDF and get some output. Right. That's sort of 5% of the problem in a way. So, yeah, we have a very mature kind of data pipeline AI ML practice, you know, that is assessing model drift, you know, and has a pretty extensive kind of test set of documents that we use. Then we built in some direct customer feedback about extractions into the product itself. So if customers agree, disagree, or agree with the data that we've extracted, they can send us a signal that we will monitor to assess our models and improve them in the future.
- [24:57]
Larry Ginn
I'll give one kind of further example of how it's actually the kind of human feedback and reinforcement is pretty naturally woven into the agreement kind of process. And what I mean by that is, if you think about that legal kind of negotiation scenario that Dan was mentioning earlier, oftentimes legal department will get a contract that some external party is trying to get you to sign. And, you know, what we have is sort of this AI assist technology that will suggest revisions, even changes in language and terminology based on that playbook concept, based on kind of your internal rules about what's acceptable, what's potentially risky. And that's an opportunity for us to iterate and present new options, new language. And then because it's a sort of legal Persona, who's looking at it, they're the ones going in, accepting those changes and rejecting it. So that's a pretty natural, you know, kind of signal about the accuracy of it. You know, almost similar to kind of classic search results, right? Sort of. Which ones do they select? Which ones perform the best? You know, was it the top result that we gave back or was it the fifth or sixth one. So that's just another example where Stan's point is. Some of it is, you know, obviously using our own golden set and being able to test for improvements or regressions, you know, F1 scores. Or it's using, you know, our users who are, who are kind of naturally giving a signal on how well our models are performing.
- [26:17]
Shawn Falconer
In terms of that feedback loop that you're getting from the people who are using this, are you able to essentially customize that at a customer level or is this like an aggregate customization that essentially my feedback might influence the model wholesale across all customers?
- [26:31]
Dan Selman
I would say we're close to the beginning of that journey than the end. So we do support custom extractions. So that's where customers are defining their own extractions. So I don't know, maybe I'm a company in aerospace, right? And most of my contracts talk about aerospace engines or something. And I have an internal taxonomy of different types of aerospace engines that I sell. So our customers can bring those extractions into the product and we will use them. That's incredibly powerful. I think we want to do more in the future, but that's where we are right now.
- [27:04]
Shawn Falconer
In terms of the testing, you mentioned having your own documents that you can test with. Are you using any sort of eval frameworks that exist for AI today or is this all stuff that you've built custom?
- [27:16]
Dan Selman
It's fairly custom. These ML pipelines are very custom and I would say sort of a sidebar. Right. Our biggest asset as DocuSign is our brand and the trust that companies place in our brand. So we take this issue of sort of data governance and the data that we use for evaluation and testing incredibly seriously. We have a very straightforward opt in AI training policy. So we will only use customers data if they explicitly opt in. We will go to great pains to anonymize and aggregate all the data. So we've had to build a lot of our own custom pipelines to support that sort of functionality at our scale.
- [27:58]
Shawn Falconer
I think that makes sense. I think that's consistent with what I've seen from most people who are doing this in production for real customers. Is it? You know, I think a lot of the tooling infrastructure is just not there yet, especially for probably doing something at the scale that, you know, DocuSign is doing this at.
- [28:12]
Larry Ginn
Even though I would add, you know, some of the industry benchmarks around, you know, kind of what constitutes good anonymization, you know, being able to recognize what sensitive information and being able to ensure that that doesn't feed into your training system. Some of these benchmarks are relatively nascent and honestly the industry ones maybe aren't as high as they need to be. When you kind of think about the level of trust that you know, your biggest enterprise customers and very sensitive industries like healthcare, financial services, manufacturing, you know, there's a very high bar and I think as a general, the kind of the industry and whether it's, you know, heuristic, like an F1 score or the definition, you know, you know, what's your accuracy and anonymization, some of this is, I think is still getting established. And I think internally, you know, we try to have a very, very high bar because as Dan mentioned, the trust in the brand that's kind of that DocuSign brand is just so important to what's made us successful.
- [29:08]
Shawn Falconer
And I think also when it comes to anonymizing sensitive data or detection of sensitive data is very context dependent. There's obviously the sort of Paris Hilton problem of Paris the person versus Paris the city. But I think it goes beyond that of if I have a contract I can have things that are particularly sensitive to my business and the people who are involved in this agreement that is not necessarily sensitive to other people. I don't know the formula for Coca Cola or something is clearly something that they would want to protect that's very, very sort of context dependent of that company. So it's a very hard problem.
- [29:41]
Dan Selman
And little side note, right? I mean, OpenAI open sourced a set of chat logs for researchers a couple of months ago. I think it was two, three months ago. It feels like years ago in this crazy AI world we're living. But if you actually trawl through those logs or search them, you will very quickly find fairly sensitive data in the chat logs, even though they've gone to some effort to anonymize them. So as you say, it is a challenging problem.
- [30:06]
Shawn Falconer
Well, I know you worked on Alexa, I worked on Google Assistant. Like I used to say, you know, when it comes to like chatbots, conversational AI, like chatbots essentially see everything. They're like a keyboard, they like. You have no idea what someone's going to put into those things. So it's very, very difficult to keep sensitive information out of it in terms of thinking about the extensibility of everything that you're building. Like a lot of this is built on, you know, APIs, developer platform, developer ecosystem. Why was it important to focus on making this something that people can essentially take these building blocks and extend and build their own Experiences around.
- [30:41]
Larry Ginn
Yeah, it's a great question. Obviously important to this audience, kind of make it a little bit more relevant. So, you know, it's interesting esignature in general, but definitely, you know, DocuSign specifically. A lot of the use of it historically came from our developers and using our APIs. Right. You think about E Signature, it's not a product like a Slack or, you know, a place where everyone goes to. A lot of the times the signing a document happens as part of a workflow that originates somewhere else. Right. So if it's like an employee offer letter, you know, it gets generated out of something like workday. If it's a sales contract that you're trying to get your customer to sign, it probably came out of a CRM system like Salesforce or HubSpot. And so developers played a really important role because they would effectively integrate our APIs or you know, we also had language specific SDKs, we also had iframe embeddable versions of that signing experience that developers will integrate into their custom product. It could be part of software they sell to others, or it could be some internal SOL they're building for their own organization. Right. And so we have a long history of that. Like I said, you know, about 50% or more than 50% of our, you know, documents that get signed, those transactions were actually API generated in some shape or form. And so looking kind of thinking about the future and a lot of the problems that we wanted to solve with the technology that Dan and I were talking about, whether it's, you know, AI extraction, understanding of agreements, whether it's helping you create and negotiate agreements. You know, it was really important that we recognize, you know, every customer has a slightly different need, you know, depending on what industry you're in, depending on what country your business is in, depending on whether you're a small customer or a big one. There's just so many different variations of agreements. The data structure, the workflow and process around it, you know, how what degree of trust and sort of authenticity of the data you need, or the identity of the signer. So there's all these kind of points of extensibility. You know, there's the need to extend our kind of UI and product surface area. There's a need to extend the workflow with custom steps and sort of data input, output processing or file processing. And there's also extending the actual semantic model of the agreement itself with like additional rich attributes. Because if you're in manufacturing, you want to be able to express key terms around quality and Tolerance and, you know, units shipped and all of that stuff. And you want to actually have those attributes in your data model for your specific agreement definition so that our AI is able to extract and recognize those. So as we kind of built out this new intelligent agreement management platform, we thought about all these different scenarios and use cases where, you know, developers would want to come in and be able to build extensions to our core products to be able to support those kinds of use cases. And so we, you know, we kind of modeled our APIs after that. You know, for example, we have, you know, the ability for you to integrate basically an external service into our workflow. You know, kind of similar to like an Alexa or Google skill. You know, whenever something happens in our system and we need to go call out to a third party system to pull in some additional data or verification of an identity. As an example, we have an extensibility model where, you know, developers can kind of build a service in a programming language of their choice, host it in a cloud provider of their choice, and then they would basically give us some metadata so that we can call into that, that external service. Almost like, again, I'll kind of use the smart assistant analogy. It's like, you know, building a skill that can plug into our, our framework.
- [34:05]
Shawn Falconer
In terms of like, from a design perspective, like figuring out what the right abstraction layers are. How did you think about that? You want to take enough work off of somebody's plate to make this worthwhile of getting value from these systems, but you also need to have essentially enough flexibility that you can support all this like context dependent either like regulations that they might have to deal with or workflows. How do you kind of like work through that design challenge?
- [34:30]
Dan Selman
Yeah, it is a challenge. Right. And DocuSign has been around for, you know, a couple of decades. So as Larry said, we have a lot of experience at solving our customers problems. And I think it is a sort of a stack approach essentially.
- [34:43]
Larry Ginn
Right.
- [34:43]
Dan Selman
We have lower level primitives, SDKs and APIs that are extremely flexible and extremely embeddable, but they come with a pretty significant learning curve. Right. For example, if I wanted to build a completely custom app and embed sending a document and signing on a web page in that custom app. We have all the tools and SDKs for you to build that custom app. Maybe you're building your new NEO bank and you want to do KYC as you're onboarding new customers and there's probably some regulatory documents that you need to sign and you have a completely embedded experience. So that's sort of the lowest level primitives. And then above that, more recently we've added features like Maestro that allow you to do a lot of those common tasks, but in a much more declarative way. So instead of writing like Node JS or C to orchestrate those tasks, I'm using boxes and arrows and I'm building a process definition. And then our process engine will run that process definition for you. And one of the steps in that process definition could be a call to our document generation API or you could call the document generation API yourself by hand. So we kind of have that sort of Russian doll approach which is really grounded in the problems that our customers are solving and their use cases. We want to make the 80% super simple, as declarative as we can. You know, five minutes to wow. Whereas we do acknowledge that there's always that hard 20% where you, you need to drop down into the code and you know, you probably need to be a more technical user to build those super customized experiences.
- [36:19]
Shawn Falconer
You know, we started this conversation talking a little bit about sort of making this analogy to like to maps and the evolution that's happened there. So, you know, if since we e Signatures is maybe the MapQuest version of this, then now with this investment, you're perhaps in the early days of Google Maps. What's this look like 5 to 10 years from now in terms of the future of agreements and their management and making this essentially cutting into this $2 trillion that businesses are losing because of poor management of these things?
- [36:48]
Larry Ginn
Yeah, well, first off, if we can get a small percentage of the 2 trillion, I think we'd all be happy with that.
- [36:53]
Shawn Falconer
Yeah, I'm not asking for 10% here, just small fraction of percentage.
- [36:57]
Larry Ginn
Yeah, yeah, we'll take a small cut of that. Yeah, I mean, I think a lot of it is around automation and you think about kind of this convergence that we're seeing between generative AI and also automation of business processes and workflows and obviously agents is kind of the hot topic these days. You look at every major enterprise company, super, super gung ho about it, right. Whether It's Salesforce or ServiceNow Workday. Microsoft, I do think, putting aside kind of the marketing hype machine aside, I think there is something real to be said about, I think the quantum leap with gen AI and the application to business process automation I think is pretty substantial because in the past the big challenge was it wasn't necessarily that you couldn't automate stuff. I mean, lots of vendors out there have built workflow automation, BPM ipaas, pick your acronym. But I think the challenge was always, you know, you need to have someone sit down and really think and map out, well, here's what my business process looks like on paper and here's the boxes and arrows and let me basically map that like into this tool and use the tool to go create the digital representation of it. But you know, it's a big effort, there's a big learning curve and you know, as much as you want to like democratize that and have, you know, Sally, who's an operations person in HR be able to do that, you know, it's pretty tough, right? People have their day jobs and for them to want to be effectively like almost like developers to go and do that is pretty tough. But I think what's really interesting with Genai is really, you know, being able to take a very, very natural language sort of prompt and here's the problem that I want to go solve and using kind of agentic reasoning and be able to figure out what steps to go execute, potentially even what external sources of truth and knowledge or what external systems and APIs to call into to go and carry that out in kind of a multi step fashion. I think it's still early, but I think that's what's really, really exciting. And if you think about the agreement process and all the things that we talked about, these are pretty complex business workflows and some of these are the most complex that exist in, you know, big corporations, even small, small businesses. And so making it easy for people to be able to automate those, all these complex processes using agents and Gen AI I think is pretty interesting. So I think, you know, maybe 10 years from now, you know, a lot of the steps that we mentioned could be pretty automated. You know, you don't, you know, that doesn't mean that there isn't, you know, work for, for lawyers and, and for you know, other people as part of the process. But I think we can take a lot of those manual kind of wasteful, inefficient steps out of the equation.
- [39:29]
Shawn Falconer
Yeah, one of the examples you mentioned earlier was the idea that there's some sort of regulatory change. For example, like India just passed, DPT is now being enforced there. So that's probably going to potentially have some sort of shrapnel damage to existing contracts. Now the state of the art today is I might have to go pay somebody to essentially review all those existing contracts and see does dpdp, is there anything we need to change here or take into account? But I can have an agent go do that work for me and then have essentially an expert in the loop to evaluate whether the output is correct or not. That's hugely valuable.
- [40:03]
Dan Selman
I plant a seed, maybe as we're thinking about the future. Right. So lawyers, they talk about this concept of a company as a nexus of contracts. Right. Company is just a set of contracts, fundamentally. Contracts with employees, with suppliers, with the government. And I think traditionally we've thought about those contracts as very static things, passive things. They're on pieces of paper in a filing cabinet. Traditionally. Right. We're transitioning now, I think, in the coming decades to a mode where those are active running entities, agents, programs, whatever you want to call them, so that I don't know, if I have a supply agreement with a company and that company declares bankruptcy, that information should enter my enterprise and ripple through my whole enterprise. That's a risk event. Right. And I think we're now able to build systems that could be much more reactive like that with, as Larry said, human oversight, kind of human in the loop. But we'll get those signals in almost real time or I don't know, there's a typhoon in Southeast Asia and it's going to impact our delivery schedule from our factory. There's. How does that impact the SLAs that we have with some of our customers? Can we get ahead of that situation, reach out to them proactively as opposed to them getting angry that we didn't meet our commitments and then suing us? So it completely changes, I think, the way that we think about the role of contracts in the enterprise.
- [41:31]
Shawn Falconer
Yeah. And in a lot of ways, these become less static assets too, that are shoved in a file cabinet somewhere, and these become essentially more of like a living, changing artifact over time. As these changes happen, updates can happen or, you know, they essentially impact business processes as there's, you know, new information available.
- [41:51]
Dan Selman
To your point, you could have an agent that's just sitting there monitoring Indian government privacy regulations. It's just sitting there doing that all day, every day. And then when it sees something interesting, it could kick off some work, you know, to repay person contracts.
- [42:06]
Shawn Falconer
So what's next, I guess, for the intelligent agreement management in the more foreseeable Future? The next six to 12 months?
- [42:13]
Larry Ginn
Yeah. Got to ask the product roadmap question to the product guy. I suppose. I think it's still early days in kind of realizing a lot of the stuff that we talked about where I think we've spent a lot of time kind of building some of the foundations over the past year. Some of the no code, workflow, automation, you know, AI powered extractions and being able to do that, you know, kind of for, at high scale, for all kinds of different types of agreements, building out, you know, also integrations to a lot of the major tools and you know, vendor software that customers typically use, like, you know, CRM systems and things like that. So I think what you're going to see this year is really kind of continuing to solve different parts of that agreement lifecycle. So we're investing in agreement, kind of creation, preparation and simplifying that. We're also going to obviously invest more in our AI understanding and extractions. You know, you could always try to add more extractions and more sort of out of the box understanding of different common attributes. But of course, you know, investing in the extensibility of it, letting customers be able to define those custom attributes and extractions, package them up and then, you know, I think a big part of is continuing to work with our developers and get them excited about what's coming. We had an event last year, late last year where we kind of relaunched a lot of what we're doing for developers and new APIs, new tools. And this year we're going to continue to do that with participating in developer events, having hackathons and then kind of showing our developer ecosystem that there's a lot more here than just calling an E sign API to send out a document for signature. That there's actually a lot of cool stuff that we're doing and exposing via APIs, whether it's some of the extractions and that semantic data model actually providing APIs for developers to be able to integrate that and maybe hook it up to their BI or visualization system. So I think there's some cool stuff coming on the developer front and then just a lot more on the core kind of product front and solving some of those problems around the agreement process that we talked about.
- [44:10]
Dan Selman
Yeah, I'd just jump in as well. The other thing that kind of keeps me fascinated with this space is that agreements are contextual. Right. The way that you sign an agreement in Japan is not the same as the way you sign it in North America. And obviously AI extractions are heavily language dependent and we want to cater to all of our customers in all markets. So you'll see a lot more focus on non North American data extraction and semantic modeling of agreements, which is very important.
- [44:38]
Shawn Falconer
Yeah, absolutely. There's today a heavy bias to Americanized English for a lot of these systems, just because that's where the original drive of innovation was coming from. But that's certainly a fascinating area and a lot of work to be done there. Dan and Larry, I want to thank you so much for being here. This was great.
- [44:54]
Dan Selman
Thank you, Sean.
- [44:55]
Larry Ginn
Thanks for having us, Sean.
- [44:56]
Shawn Falconer
Thank you. Cheers.