← Law and Candor
Law and Candor
Feeling Disillusioned with AI? You’re Not Alone
00:22:31
In the ever-evolving landscape of technology, GenAI has emerged as a transformative force, promising to revolutionize various industries, including legal. As with any groundbreaking technology, GenAI follows a predictable pattern. A common framework is the Gartner hype cycle. What can the hype cycle teach us about getting to the real value of GenAI? Fernando Delgado, Head of AI & Analytics at Lighthouse and PhD from Cornell University Bowers College of Computing and Information Science, has studied and developed AI solutions, giving him a unique perspective on these hype cycles. He shares keen observations on where we are with AI in legal and eDiscovery and how to navigate the inevitable disillusionment. His practical advice on finding applications that provide real value can help us all ride this fascinating wave to a better future. This episode's sighting of radical brilliance: "Great Strategy Starts with Experimentation," HBR on Strategy Podcast. Learn more about the podca...
Loading summary
Transcript33 lines
- [00:11]
A
Hi, welcome to another episode of Lawn Kander, the podcast exploring the latest in innovation and technology in the legal industry. I'm Brooks Thompson here with Sarah Barsky. Hey, Sarah.
- [00:20]
B
Hey, Brooks. How's it going?
- [00:22]
A
The sun's shining. It's a great day. Another day in ediscovery. We're exploring a really interesting concept, the technology hype cycle, which we'll get into with our guest, Fernando Delgado later. But, Sarah, I'm curious, looking back, what do you think the most overhyped technology has been?
- [00:39]
B
So I have two answers. And as with all of my answers here, these might just be because I am not at the cutting Edge, but my two answers are 3D printing and virtual reality. These are two technologies that have been very hyped at various points in my life, and I just feel like they've come to nothing, at least as far as I'm concerned. What do you hear about 3D printing anymore? And I bet there are many, many corporations that actually use 3D printing for something, but I just don't know what it is. How about you, brooks?
- [01:10]
A
It's hilarious. 3D printing crossed my mind too, but I actually scratched it off the list. The number one most overhyped technology, in my opinion, is cryptocurrency. Right? I mean, do you remember when everyone was like, bitcoin's gonna change the world. You'll be paying, you'll giving the pizza delivery person a tip with bitcoin before you know it. And it's like, I've never owned it. I feel pretty secure in my financial standing, I think, with all these coins going up and down and being created and worth nothing the next day. Like, what a total overhype that was. Unlike some of these overhyped busts, though, we're already seeing the transformative force of Gen AI play out in many areas in our lives and across industries. As with any groundbreaking technology, it follows a predictable pattern. A common framework is the Gartner Hype cycle. To understand what the hype cycle teaches us about getting to the real value of Gen AI, we spoke to Fernando Delgado, head of AI and analytics at Lighthouse and a PhD from Cornell University's Bowers College of Computing and Information Science. He has studied and developed AI solutions, giving him a unique perspective on these hype cycles. And he shared some keen observations with us on where we are with AI and legal and ediscovery, and how to navigate the inevitable disillusionment. His practical advice on finding applications that provide real value can help us all ride this fascinating wave to a better Future. Before we jump into our conversation, we have another sighting of Radical Brilliance, a podcast from the Harvard Business Review on strategy.
- [02:41]
B
Great.
- [02:41]
A
Strategy starts with experimentation. Sarah, what did you think of the podcast?
- [02:45]
B
I really enjoyed it. So in this podcast, Harvard Business School Professor Stefan Tomka challenges the traditional reliance on intuition and experience in business decision making and advocates for a culture of experimentation. And as with almost all of our sightings, I learned a new term, the term hippos, which is highest paid person's opinions. So one of my key takeaways from this podcast was that experiments outperform intuition. So, as we all know, many companies rely on opinion to drive decision making, intuition to drive decision making, as opposed to a program of experimentation. And often the decisions are made by the highest paid person in the room. So that was a real takeaway for me. I'm going to think a lot going forward in the meetings I'm in about how decisions are being made and about whether there's an opportunity to apply an experiment to drive decision making. Brooks, what do you think?
- [03:41]
A
I couldn't agree more. I feel like often in business settings it's like, well, what does the hippo think of this? And not, hey, have we tested these ideas? Have we gotten validation from the market or our clients? So I also am really excited to see what additional ways I can incorporate experimentation into the business decision making process. Something that we've done here on the Spectra team is client advisory boards, right? Experimenting with ideas, roadmap prioritization, showing people potential access to technology early through UAT to get feedback. And I think it's proven that it creates a better product. So I'm continue to try and use it in other aspects of my business and personal life, including what I'm going to feed my kids for dinner. A little more experimentation there.
- [04:29]
B
We have our conversation with Fernando up next and. And we hope you enjoy it as much as we did. Hi Fernando, welcome back to the podcast. It's always so great to have you. Before we dive into navigating the trough of the technology boom, it would be great to hear about any new developments in Genai that are exciting to you. Since you last joined the show, there have been some advancements and we'd love to hear from you. What will have the biggest impact?
- [04:59]
C
Well, there are a lot of things that have come up since the last time I joined and it's great to be back. I think one of the things that has really jumped out, especially with respect to generative AI, has been a new solution that we have come out with in terms of automating the issue coding process and while doing that also providing thematic analysis on documents. As you know, we do a lot of classification work using predictive AI and we've also done privilege, but we've never done anything downstream of that that didn't have to do with sensitive data like PII or source code analysis. But now we do have something to help with that immediate second pass review and that's basically on the set of responsive documents that have been identified either by humans or by machines or some combination of both. We basically provide a topical map of that responsive set. So we issue code documents in that responsive set and then we go even further and this is really just, you know, thanks to the power of Gen AI and and other supporting analytics, we're able to provide what we call themes. So think of themes as sort of sub issue codes. So you can have like a broad issue code like marketing, but within that issue code you'll have multiple themes. They could be like marketing around specific products. They could be certain concerns that people are voicing in the document population about marketing more generally in, in, in the company or competitive threats. And so this is, this is great because it's providing, like I said, a map and it's also something that we're doing on incoming inbound productions. So it's also a quick way to get a sense of, you know, what it is that you're getting and then be able to do some review and strategic review and strategic sampling of those documents to get even more insight into what it is that's in your document set.
- [07:02]
A
It's never shocking to hear how quickly the Gen AI is updating and how quickly it's evolving. So thanks for sharing that. Although all of us have certainly witnessed this both in our personal and professional lives. Can you explain to us what a technology hype cycle is and why it matters for legal and ediscovery?
- [07:19]
C
Yeah, I think intuitively we all get a sense of a technology being overhyped. But the hype cycle, and I think there are a few different versions, but the one that I'm most familiar with comes from the Gartner, the research firm and how they think about technology and its adoption is really sort of ups and downs. Right? We're in the beginning, there's targeted publicity, right. You might feel like it's the only thing you're hearing about and then there's some early success stories and that this really feeds, you know, a lot of speculation and a lot of excitement right around that new technology that's coming in. Especially by, you know, that early minority of there's always going to be champions and tech enthusiasts, and so they really feed into that hype cycle. But often, and this is just, I think, the human part of technology, all of that excitement leads to expectations that probably can't be met, Gartner itself calls it. You reach a certain peak of inflated expectations, right? And so naturally, once you get to that point, there's no way but down, right? And then that next phase after that peak of inflated expectations and probably that great deal of speculation, you go into what they call, poetically, the trough of disillusionment. And it's in this trough that, you know, basically it's almost like a backlash to all that hype, right? And it's like, oh, this new technology is not really everything that it was sold to be, you know, and a sinus, you know, skepticism, maybe even a cynicism creeps in there. But the Gardner hype cycle allows us to not fret too much. You know, this is a natural part of that backlash to the hype cycle. And then after we get through that, like, there's a scope of enlightenment is what they call. And this is really where post hype people are actually learning what they can get from the technology, and they're integrating it in a programmatic and scalable way in their daily lives. And this goes for, like you said, program personal, you know, tech that changes your life personally as well as business tech.
- [09:36]
B
This roller coaster that you're describing of hype Cycle, it seems so unproductive. What do you think drives that humanity? It's a good answer.
- [09:47]
C
I think it's natural. I think it's, you know, I've learned not to judge it as much as describe it. Right. I think they're onto something. I think that it's how we process new things as a group. Right. And so everyone has kind of a different relationship, I think, to the different parts of, of the life cycle. But I think it's. I. I honestly think it's our way of getting to know a technology, right? Like perhaps it shows, like, initially we're quite optimistic and, and get really excited about something and then, you know, end up setting up expectations that are really hard to meet. And of course, this is all in the context of a lot. You know, with some of these technologies, there's a lot of money behind that. So the publicity in the beginning can be really intense, and there could be very specific vested interests in the beginning to make sure it's successful. And, and, and I think in some ways, that backlash is healthy in that it means like a larger set of people are providing input into the picture and then, and then you kind of get, you know, a synthesis at the end. I actually think now that you asked that question, kind of that hype and then like reaction to it is, is. Is a productive part of getting to know a technology. So. But it's painful, so don't, don't want to ignore that part.
- [11:11]
B
It's funny, it actually kind of reminds me of something that we talked about last time you were on the show, which is a lot of people not wanting to be left behind when something is new, but. But then not really knowing what they want to do with it. And so they approach it without a plan for what to do with it, and then it lets them down.
- [11:28]
C
I think that backlash also has something to do. I mean, there are certainly people who are always skeptical about new things, so that's part of it. But I think you're right to point out that that's not the only thing that's contributing to it. It could be. It's just almost an immaturity. Again, not judgmentally about thinking about how to integrate it. We're still learning about this new technology and what it. And what it could do and that. And that can be frustrating.
- [11:57]
A
It feels like what you're talking about here is users being disillusioned with the prospect of AI and not getting out of it what they thought they would get out of it. So is there any advice you have of overcoming that potential disillusionment to fully take advantage of the benefits of gen AI?
- [12:15]
C
Yeah, I think, I think a big thing is really getting to, from a business standpoint at least, like the specific pain point, what is it that you're either trying to fix or transform and having that be the anchor. Like, in a sense, I think the advice is, you know, we're not, we're not adopting technology for technology's sake. I mean, sometimes we are. And I think, I personally think that when we do, you have to kind of question if that's the right thing. It's more about like, what is it helping you solve? And I think once that helps you be more specific because usually that's something you know more about. Like, I think one of the challenges is when you're trying to just think about the technology and abstract as opposed to it being something that's helping you in a specific part of your life. And when you turn it into, what is it trying to help you solve? Well, you're an expert in that area. Like, you're the one who. It makes it into something more concrete. And the success criteria being something that, that's like, not abstract about the technology, but about what it's affording you or what it's affording your business. And that, that to me leads to, I think, a more productive, like what I like to call, like an engaged skepticism. So I think it's good to be skeptical about technology adoption, but at the same time engaged. Right. Not letting that skepticism paralyze you. And then I. In that mode, you know, you're kind of, you're, you're a tough audience and you're kind of hard to please. And when you do see success, you're really learning from that. And then I think that leads to just, just a better use of technology and, you know, a better understanding of how I can make things better.
- [14:07]
B
You're kind of walking us into the next thing that we wanted to talk to you about, which are the risks and traps of the disillusionment part of the hype cycle. Where do you see that come into play with new technology?
- [14:18]
C
I think there are multiple risks. One risk is, you know what I was saying before, like getting paralyzed, being so overwhelmed with the new that you don't even start and you of go into a passive mode and you just want to see other people's successes before you feel it's your time to start experimenting. In some cases, in some cases that might be the right decision based off of your personal goals or your business model, but in other cases that means that you can be left behind. Having a plan to start early I think is key. But then also being realistic and understanding this hype cycle somewhat inoculates you, inoculates you from it. So it being prepared for that trough of disillusionment, I think is, is important because it's going to happen. There's going to be a period where the hype and the excitement, you know, people can be impatient. And when it doesn't materialize into benefits, you know, that's really where the real work happens. Like doing that steady work of experimenting, seeing where it works, seeing where it doesn't work, establishing that knowledge and then pushing through. Those are, those are the people who are really surviving that, that cycle and are going to use technology, whatever that emerging technology is here for us. You know, it's AI they're the ones that are going to be the most successful and the best at it.
- [15:46]
B
What about the risk of starting too small?
- [15:50]
C
Yeah, that's, that's interesting. You know, like our. A common sense view holds like, you know, it's always better to be, let's just be incremental about this, you know, let's get the low hanging fruit. I think the issue with that being at least your exclusive approach is that you're really not seeing the full ambit of the potential of a, of a new technology and you're really not pushing yourself or your team to really treat this as potentially transformative. So while, you know, working on smaller projects to see if it's almost sort of like a kindling to like a, you know, more dynamic fire of innovation downstream, often just the fire doesn't really start in that way and you sort of get an early, but perhaps, you know, low adoption and not really, not really giving you space to transform the way that you work.
- [16:56]
A
It feels like some of this disillusionment that you've been talking with us about is a result of people expecting this massive ROI in the use of this technology and then not being able to quantify that. So what do you think are some tips and tricks about thinking about measurable results? And what kind of view do we take on roi? Is it broad, is it specific? Is it allowing us to do more work that we think is important? How are you seeing that disillusionment be overcome through a better understanding of the ROI that Genai can offer us?
- [17:31]
C
I think often the inflated expectations come from a sort of mentality that, you know, a, it's going to be push button and easy and then that it's going to happen all at once. So if it's an efficiency goal that you're going to replace human labor like easily and in one step and that's, that's just not the right perspective to come at it. Like, even though the end product of integrating technology can be transformative, it, it's going to take a few steps to get to that. So yeah, one, one sort of inflated expectation is, is actually the speed of change that this is going to happen across. And so that, that's something to keep in mind. I think another thing here is not really being clear about what you want to change and then having that. I've seen clients have that light bulb moment where they go into it thinking really they want to replace as much human labor, they want to use AI in particular here to automate. But as they get to know the technology more, what they realize is sure, you know, there's probably going to be less human effort when using AI, but what's really interesting here is that we actually can do now a deeper analysis than we could have before we could augment the humans who are doing this work. And then the question comes into less about efficiency and more about, well, what's this overall experience? How does this change? How could this change people's roles? How could this change how we think about work more broadly? And that type of analysis is harder to incorporate into an roi, but I think it ultimately makes for a much more robust ROI analysis because you start to see not only things from a task level like, this is how I can automate this task, but it's like, oh, this is how I can change how I even think about work or our business or, you know, how we think about different roles in the organization and those more, you know, quote unquote, softer things are just as important as the easier to measure efficiency gains. So I think a key thing about ROI is, yes, efficiency is important. Cost savings are important, time savings are important. But you have to measure other things in terms of quality of experience that the AI is giving you. And just the new types of analyses you can do that you couldn't do before because you have AI.
- [20:16]
B
We are currently living in a moment of the Gen AI hype cycle. What do you think is next for us in that cycle in our industry? Fernando?
- [20:24]
C
I think the way that the hype cycle is manifesting itself broadly right now is probably, and it depends who you are, we're probably peak of inflated expectations. Some of us in the trough of disillusionment. I don't, I, you know, maybe a few of us in the slope of enlightenment. But this is interesting, though.
- [20:47]
B
The cycle isn't the same for everybody. We all are going through it at different moments.
- [20:51]
C
I think we're all going through it at different moments. You know, like the people who have, the early adopters are going through that cycle, you know, before and it, and it, and it probably feels different depending on where, you know, if you're experiencing this with a bunch of people or a small, a small cohort. But I think mostly we are probably falling into the trough of disillusionment. This was, this was my take a few months ago. I think it continues to be my take. And I think part of that is we're talking about Gen AI often very broadly and not in the context of use cases. And that to me is like a symptom of inflated expectations. We're just saying, like, it can transform everything. And now people are realizing that actually maybe it can't transform everything and that. And that's taking us into the trough of, of disillusionment. But we do also see that. And that's why I think we may be in the trough, but some of us are pushing a little ahead is that we're getting very specific about how it's helpful. And then those are almost like beachheads for and us just getting to understand, like, what its strengths and weaknesses are like any technology.
- [22:08]
B
Fernando, thank you so much for joining us today.
- [22:10]
A
Really great to have you back. Thank you.
- [22:13]
C
Great to be here.
- [22:15]
B
Thank you for joining us for another episode of Law and canon. Visit lighthouseglobal.com to continue the conversation on AI and rate and subscribe to the show wherever you listen to podcasts.