Loading summary
Transcript119 lines
- [00:01]
Sponsor/Ad Voice
This episode is brought to you by Indeed. Stop waiting around for the perfect candidate. Instead, use Indeed Sponsored Jobs to find the right people with the right skills fast. It's a simple way to make sure your listing is the first candidate. C According to Indeed data, Sponsored Jobs have four times more applicants than non sponsored jobs. So go build your dream team today with Indeed. Get a $75 sponsored job credit@ Indeed.com podcast. Terms and conditions apply. Want to see your brand on tv? Roku Ads Manager makes it easy to launch targeted ad campaigns in minutes, track results in real time, and drive on screen purchases with just a click of the Roku remote. Get a $500 match on your first $500 spent with code ROKU500@ads.roku.com that's code R O K U 500@ads.roku.com Terms apply.
- [00:53]
Matt Russell
I think that's really where it comes into play is giving any individual investor a ton of leverage in terms of their traditional work process and due diligence process. You can put in a monitor for any type of interesting commentary around logistics coming from okay, let's pick out the sectors that matter most to these specific companies that I care about and you can input that and set it up so that it's done on a reoccurring basis and sending you information that otherwise you just would have never seen. The best people that I know at using AI have very specific systems which anytime a new model is released they'll have their prompts which are detailed and they'll see what this new model does in terms of output and be able to compare that to the previous model. I still don't think it solved this really obvious alpha generating solution. Most of this is just coming out as an extension of what they might have otherwise done, but they're able to do it faster.
- [01:59]
Matt Sigler
You're watching Excess Returns. I'm Matt Sigler. We're talking AI, how to use it, how not to use it, and I can think of no better guess than Colossus and Business Breakdown's Own. One of my favorite investors, Matt Russell. Welcome back to Access Returns.
- [02:14]
Matt Russell
Always a pleasure to be with you and you make me brush up on my communication of whatever is going on up here in my head. So it's always a good excuse to get on and talk through things. So it's good to be back.
- [02:28]
Matt Sigler
We wanted you specific to this because you did a Business Breakdowns episode that was walking through using AI as an investor, an allocator, an advisor, all these things and both Jack Forehand And I were behind the scenes going like, this is a really freaking cool episode because you unpacked. It's soup to nuts. So we want to kind of take that and unpack it, broaden it out. I'm putting you in the hot seat as opposed to say, say the who the guest was on that episode of Business.
- [02:54]
Matt Russell
So that was David Plan, who runs Portrait analytics, which is an AI business. And I will be 100% honest. David is somebody who is always probably six, probably nine months ahead of me on the curve. And he has a great way of just seeing where things are, where things are going, and it helps me as well.
- [03:16]
Matt Sigler
Okay, good. Well, he can be nine months ahead, you can be four to six months ahead of me, and now you're going to catch me up. So I'm taking it all the way back, timeline style, how we got here, the early GPT. I still remember whatever it was two or three years ago, seeing these prompts in a conference room, basically with somebody who was like a couple of years ahead, who had just quit his job, was like, this is the future, here's why. And I'm like, is a Google search better than this still though? Can you clarify? He was like, kind of. But no. And I'm just curious, like, what shifted when models started to become actually useful in research?
- [03:50]
Matt Russell
Yeah, I think that's honestly one of the head fakes for investors specifically, is that when you use those early models. ChatGPT, when it came out, it was this social experience in many ways. And yes, it was part Google search, part communication tool. But once you started to get into how you would apply it as a professional, it showed a lot of the cracks that existed in terms of where those models were. And I think a lot of people left it behind in those early days. Now they're catching back up. But where the big inflection I think came was when the deep research models started to come out. That's late 2024, early 2025. This is what coincided with these premium tiers of each of these AI models. You started to work with deep research, which we can get into what exactly that means. But it was more like agentic AI than a traditional LLM. And I think that's when the model started to show some understanding of exactly what you were trying to do and what was valuable within that. And that's where it took Step up from being this fun little social. I can use it in personal ways to something that I can actually apply as a professional investor.
- [05:11]
Matt Sigler
Okay, so this is something going through your stuff that made me just sort of reflect back on the experience again. From sitting in that conference hall and seeing a presentation to playing around with it and going, okay, you said it could do this, but it can't really do that. And then having somebody smarter going like, well you're right, it can't actually do that yet, but it's going to be able to. I'm like, cool, can you tell me when it's going to be able to? You talked about LLMs versus agents. Explain the distinction, explain why this matters.
- [05:39]
Matt Russell
Yeah, it's a little bit of a blurry definition and I am definitely not the right technical person, but maybe I could be a translator of everything that I understand from the technical sense. The LLM in a basic free tier sense is like a calculator. You're going to give it a question, it's going to take one shot at that question and only that question by basically searching simply. We can think about this like a Google search of if I ask it to look at a nuclear company's growth prospects, it's going to bring me whatever that nuclear company said in their investor presentation and maybe one industry wide example that's out there. If I go into an agentic workflow, that's something that is now using reasoning to try to get at the heart of what you're asking, understand that there may be layers to it. You're asking about this company specifically. You might also want to know what is going to drive that growth, whether there's cracks to what they're saying in their guidance. So it's going to present to you something that is much more in depth on the topic using a much wider variety of sources. It's going to take more steps in coming to the conclusion, more time and it's going to give you something back that you can actually use that looks more like what a junior analyst or associate would do than an intern would do, which is what the free tier LLM is.
- [07:13]
Matt Sigler
The free tier LLM as intern is fascinating in and of itself. This, this idea of it inside of the workflow from an invest from an investor. So put your investor hat on like strictly in this and I'm drawn to it from what you said before about the calculator step versus the actual like reasoning and logic and having above an intern's level of understanding here. Where does it fit inside of a workflow? Just at like a high entry point level for somebody who's never encountered this before.
- [07:44]
Matt Russell
Yeah, it can take a variety of different tasks on depending on where you look at the workflow. So if you're monitoring your existing positions, you might like to do a quarterly recap of all the key Data points, key KPIs in your universe and with agents or just AI in general, you can set them up such that you are going to save a massive amount of time. So you go back to Matt as the sell side analyst covering transportation. It was the trucking earnings and I needed to know, you know, what customers were saying about logistics costs on their earnings calls. The customers are not in my coverage universe, which meant I was control effing through a ton of transcripts, spending time late on Thursday nights at the end of earnings season while everybody else is out enjoying happy hour in Manhattan. And that just goes away. So you could do a lot of these things and they could be ad hoc. So I can set something up where I do that research after earnings season ends, or I can do it where it's set up such that it's monitoring those on an ongoing basis, essentially like data scrapers and sending me alerts on somewhat of a regular basis. But it's much more tuned to what I'm actually looking for. So the previous world, it was a Bloomberg alert for anything that comes out on ExxonMobil and I'm going to get a headline that they won the 2024 Louisiana Manufacturer of the Year along with a million other emails about ExxonMobil in my alert inbox. This is going to be much more specific and tied to what I actually care about. And it can extend the universe beyond what you would tell it to initially look for, which is where the agentic reasoning comes into play. So that's just one example from a workflow perspective on the end of it, but then also on the due diligence phase. I think, you know, a lot of time was spent initially getting up to speed on names by looking for sell side primers, which are usually outdated by, you know, a year or at least several quarters waiting to get those back, trying to synthesize them over time. And with a lot of these deep research reports, you can have those off the shelf and ready fairly quickly. So there's a productivity uplift which is very visible immediately today.
- [10:11]
Matt Sigler
Is this something when you started playing around with it, you immediately looked back at prior roles that you filled and you were like my life, however many years ago no longer has the need to exist. But you gave the example of missing out on the happy hour because you're going through these reports and earnings releases. Everyone I know who's been a junior analyst has or A banker has a version of this story.
- [10:32]
Matt Russell
Yes, 100% now, did I need to exist? I like to believe that I just would have been better at my job and spending more time on the things that mattered. But yes, I did.
- [10:42]
Matt Sigler
It's a Wonderful Life. Matt Russell Edition. I can only imagine. Don't take that the wrong way.
- [10:48]
Matt Russell
No, no. I think where it really comes into play again is this analogy which I keep coming back to. If we looked at AI as, you know, your intern several years ago and somebody who's coming up to speed in the organization, you usually dedicate the most time and put the most energy into those people that show the most potential and that continue to get better and you focus on them and keep developing them. For me, that is where AI fits in the most. So AI is better than I was as a intern, for sure, and much faster than I was as a junior analyst. And there's certainly things that it still can't do that I was able to do, but that gap is shrinking massively. And I think that's really where it comes into play is giving any individual investor a ton of leverage in terms of their traditional work process and due diligence. Process work, due diligence.
- [11:48]
Matt Sigler
And I just want to laser in on one more aspect of this to just. I need you to flatten this out, iron this for my brain, because I've only encountered this in the last couple of months where people are using it for the real time monitoring. And that's the part that goes from I have to take this myself and drop it in to now going like, oh, no, you're going to pay attention to all these things that I told you to pay attention to and now keep me abreast in the way I need to be kept abreast. Are we really, Are we as. As early to that as it feels like for me, or is this way more pervasive than I'm aware of?
- [12:23]
Matt Russell
I still think it is fairly early on that in terms of who's doing that in an effective way. I think where many people have gotten excited is the ability to scrape all of these things, get downloads, get them sent to you on, you know, whatever basis you decide on. Daily, weekly, monthly. Where you run into problems is scraping comes with a lot of legal restrictions and compliance restrictions and API, you know, requirements that. That come into play. But yes, where it's definitely gotten better is you can get things digested which you couldn't previously get before, and the synthesis that comes alongside of it is typically better. There's obviously a question of what do I miss if I'm getting this synthesized? But based on what you were previously dealing with, which if it was a KPI dashboard, whatever it might have been, that has gotten significantly better. I don't think it's been adopted by the masses by any means, by even the average people are all over the map in terms of how much they've adopted AI but it's definitely possible and you don't have to look too hard in terms of doing it.
- [13:34]
Matt Sigler
Do you think I'm going to zoom in one more step before we zoom way back out again? Like the end game for this is it just one guy could be running the fund and have a bunch of agents going around doing like what's, what's the end game for the average professional investor with this? Does this actually consume most of what they do?
- [13:53]
Matt Russell
I think where it becomes the obvious answer today is that every team could run leaner. Some of the processes and procedures can now be done instantaneously. So whether that is at the junior analyst level, at the associate level, at the VP level, that's up for debate. It's going to depend on if it's a deal driven organization in the private world that requires a lot of in person interaction, time spent with people and one person can't do all of that. So you still need the people. I don't think people completely go away, but there is leverage created across the system. And I do think at the end of the day like the decision making still comes down to the humans. You're going to get presented with a lot of information, you are still going to be making the decision which has always been the case unless you go into the quant front territory which this stuff has kind of been around for a very long time in the quant fund spectrum. So I think it really comes down to lean leverage. Productivity is where it shows up the most and you can just cover a lot more breadth than you were previously covering with this.
- [15:16]
Matt Sigler
Okay, so I go to a private investor, I go to somebody like yourself who has heard the words but hasn't encountered the tools yet. Immediate leverage points. Like if I were to get the we're splitting your life again. I go out and I find Matt Russell who hasn't discovered any of these tools or hasn't started to ask questions or play around this yet. He's got all the knowledge of you pre this. Where do I say you have to try this? You have to start here.
- [15:42]
Matt Russell
I do think the easiest way is to just journal out your process when you're doing Any type of research, so. Or if you want to change your research process, how do I do it? What leads to good decisions? And honestly, just keep track of it in a day. And you might say to yourself, you know what, I have a hard time screening for names, therefore what is my screening process? Do I look for certain types of companies? Can I use AI to come up with screens to do that on a somewhat regular basis? Find any way to get that into a system that it spits out, so that's maybe more advanced. The easiest way probably is just testing the LLM. Pay for a more premium tier. If you haven't used it before, just look at the communication potential. And if you do any type of marketing, any type of fundraising, it can make your communication and delivery much stronger. Then move on to the research process. Okay, what takes the most time in research? Can I make that faster using AI through just the LLMs? You know, pay for a higher tier. The next step is maybe using some of the off the shelf tools which are going to come at a higher cost but are specific to the investment industry. That's going to help you monitor certain thematic things. They can be really helpful with that and give you better responses. So I think those are kind of the key categories across the board where you can find immediately immediate leverage. One of the bigger problems, and this is something that David, you know, from Portrait has talked about a lot, is investing. There's, there's a decision made at the end of your process. It's not like, you know, a medical diagnosis or coding where it's a very clear yes or no, you know, conclusion. There's, there's art to it. So everybody has a different process and that's why you really have to customize it to what you're doing to get that uplift and leverage.
- [17:45]
Matt Sigler
So after you were to work through these steps, and I mean, I can already see the, from that example, I'm going to plug in my portfolio, it's going to tell back to me, you know, you have cancer or something, I'll figure out the way to short circuit this thing. I'm confident in that. You guys talked about in the business breakdowns episode, specifically this monitoring piece and you brought this up before as a major pain point. So like in that process, if somebody hasn't encountered this yet, why would you explain, explain to Matt Russell three years ago without this knowledge what this monitoring capability is and why to be interested in it?
- [18:19]
Matt Russell
Yeah, so I think I'll take either the energy sector or again, transportation is a good one because it Involves so much of the economy. And what I would have to do to understand some of the drivers of transportation were to understand the entire economy. I could either just listen to the companies and take what they told me or extend further, learn from their customers what they're saying. If it differed from what the trucking or rail or UPS and FedEx were saying, you know, what's the mismatch? And before that just took an extreme amount of time that I just didn't have. Because you're sorting through a lot of information that has nothing to do with transportation. You know, if I'm looking up CPG companies, maybe 100th of the output from them is going to be about that specific dynamic. And this is where again, off the shelf tools will be a lot better at doing this than trying to do it through your LLM. But you can put in a monitor for any type of interesting commentary around logistics coming from. Okay, let's pick out the sectors that matter most to these specific companies that I care about. And you can input that and set it up so that it's done on a reoccurring basis. And sending you information that otherwise you just would have never seen. So that can just keep you up to speed and get and, and not even up to speed, put things on your radar faster than they were previously coming about. And it's going to, you know, stop you from not, you know, missing what was an obvious step change in terms of the environment. Because you were just listening to the companies and their bias towards, you know, the outlook being, okay, I love this.
- [20:08]
Matt Sigler
And now flip this around to the other angle too, so it can let you take more in that might be useful where you're taking it in to be useful. But I also feel like, and you mentioned this in the, in that episode too, of it can get you to saying no or shutting something down faster. Talk a little bit about that.
- [20:25]
Matt Russell
Yeah, so killing ideas, that's something that, that I'll reference David again. He mentioned it took him a lot of time to get to a point where he killed an idea and that had to do with his process. But there were certain things that might be killers, it might be accounting irregularities, management compensation, their incentive structure and how that related to the actual business and the stock. Were they being incentivized by the right things? And usually you have something that you're very specific about. It might be, you know, management having a very good track record through certain periods of time. It might be the incentive system within the organization. These are things that you can kind of Set up such that the AI knows to look out for it. I can set up a template with a deep research model that knows all the things that I care about. These are themes that I care about. These, you know, businesses that are mission critical in terms of what they manufacture, but represent a small percentage of the overall cost base of whatever, you know, the output is that they're selling to. That means they have pricing power. There's a million businesses, all in tangential industries. It would be almost impossible to cover all of those things. So we can simultaneously screen for those things, but also look for things that I know are deal killers and highlight them to me, allowing me to cut them off earlier in the process. And that again gets into productivity iteration. Just the speed that you can go through things. As a private investor, I think that, you know, you don't get nearly as much information, but sometimes you're on the clock. You're looking at 10 different deals that hit your desk every day. Do I want to even entertain this? You could scan for those things, set something up such that it's going to at least highlight it to you very quickly. And you know, okay, this is a major hurdle that I have to overcome. Or I could just write it off immediately.
- [22:22]
Matt Sigler
The trust but verify situation of this too. Once you set them up, once they're running, how do you stop? Or do you stop to pause and say, let me review the source inputs, the prompts, the like, make sure it's still doing what I think it's doing. Because it's very easy to do this, see it work once and then go like, I never have to think about that again, but doesn't feel like that's the case. Clarify.
- [22:44]
Matt Russell
Yeah, this gets into something with prompting and how important it is. I try to make sure in every prompt that I set up as it relates to anything research oriented is if I'm asking it to take a view on something, use any type of projection, ask for the sources that come along with it. And you could just ask for sources in general. Where are you getting these numbers? You know, put that in the prompt. Please include your sources as it relates to certain data points. That way you can check through and scan to make sure that it aligns. And I think the hallucination problems have gone down significantly. You can see when you are using these LLMs that you can notch it up, you know, in terms of how much flexibility you want to give the model in terms of what it comes back with. If you want them to be creative and iterate a lot and kind of Use their own reasoning to come up with answers for you. You could do that if you want to notch it back and just use kind of like a straight line approach to it. There's going to be less reasoning, less chance of hallucination. So there's multiple steps that you can take to safeguard against that. And I think ultimately what you're trying to do is get to a similar point that we're at with other humans. You know, you ask analysts and associates, you know, potentially to help you out with certain things. I've been on the other side where I got a number wrong in a model and we published it and then it had to be rejected. So it still happens on the human side of things as well. You just want to create as many safeguards as possible where you can double check the work. Once you know that it's running comfortably, you start to, you know, ease up on some of that. And you can just again, ask the LMS for the sources when you send the prompt.
- [24:36]
Matt Sigler
All right, let's get into the down, down and dirty gritty details of what's in your tech stack for these. And on that last point, too, I feel like everybody, you have to have the moment of where it terrifies you on either the hallucinating or something going horribly misinterpreted, or it says something back to you and you're like, oh, my God, I've angered the robot overlords. I feel like that's an important experience. I had that with perplexity. You've experienced this, clearly?
- [25:05]
Matt Russell
Absolutely. You can understand where it hits issues. Sometimes you'll ask for things and it tell you, tells you you can't provide it, and you're wondering yourself why this doesn't seem that. And then you can ask it why it can't provide it, and then it'll specify and you're like, oh, well, I hope I'm not on a list now. I didn't even think about that. But there are all types of things which pop out at you, which you understand how the models work. I think just seeing how these models work is useful. And they're changing so quickly, so you can't get too anchored in terms of whatever happens. And I think that especially early on with going all the way back to the beginning of the conversation, a lot of people became anchored in those early LLMs which were much less capable of helping you out. But I do think the only way to see progress is to be experimenting and seeing, okay, now it can do this. It couldn't do that before. So that is important and you'll still have those moments. I don't know if we passed the five finger problem yet, but that's one that you know is always top of mind for me.
- [26:12]
Matt Sigler
Always top of mind. Make sure you get those finger counts correct whenever you can. Which so what what models like what are you using now? How many different things? How many like more generic public things are versus private fancy things? What are you using? What's in the Russell stuff?
- [26:30]
Matt Russell
I have I am not monogamous in any way in my AI usage and test a lot of things, you know, not. I would never portray myself as the AI in the weeds on every technological advancement, but I do like to experiment with all of it. So for me, Claude for writing because Claude was the best with writing assistance editing better analogies. Let me frame this in a more concise ways. Let me say this in a way that's not extremely condescending. Let me set up a project that writes in a very specific tone for this specific use case. So Claude has been just kind of the go to writing and that's actually had stickiness for me in ways that others haven't. So I would mention that as well. ChatGPT is kind of my do all universal model. I will use the deep research from time to time. But also if I'm just doing anything generic really from a personal sense, GPT will be the one that's. That's my typical, you know, use case Gemini for deep research. I've gone back and forth. I've used Chat GPT for deep research more recently. Just I like the quality more so that. And then the last thing I would mention which has been surprisingly valuable is I use DIA as my brow DIA as my browser. Excuse me. And what that has allowed me to do is if I have a Google Sheet open, I would imagine this is like copilot for Microsoft users, Windows users. But if I have a browser open with a Google sheet, I can just chat through some of the scenarios that I'm looking for inside whatever the model is or the presentation of the data for me. And it can do massive amount of things. It just saves you a step from having to upload documentation, upload a website, upload text and it can do quite a bit with that with email, anything that you want it to. So again, I didn't give you one strict answer, but all of those models are things that I use quite a bit and then playing around with the code models, but that's. I'm much lower in terms of the advancement curve in that sense.
- [28:48]
Matt Sigler
Inside of that, and especially with this conglomerate approach, you've also talked about sort of encoding mental models, your own frameworks, heuristics, shortcuts, patterns of thinking, like into both the prompts and the systems that you're using. How do you think about that?
- [29:03]
Matt Russell
How do you reconcile that it's actually good? Because it answers your other question in terms of more of the advanced, expensive tools. That's where something like Portrait comes into play. You could look at AlphaSense, there's other AI tools out there made specifically for investors. And what that can do is it's going to have a much more strict criteria in terms of where it's searching for the information. I can have mental models in my head, which I still need to be able to articulate in a way that's understandable to anyone, to someone like you. If I can't articulate it to you, it's not going to be articulated to a model. But that I think is really important because most of us have mental models that drive, you know, our investment decisions in some which way, even if we haven't figured it out yet. If we say that we're, you know, our pattern matching, then I think that, you know, it suggests that there is some mental model in our head for what we look for. And that's where those tools really come into play. They can be, you know, linked into the alarms as well. Where, because there's memory now, you know, you could put it in your profile in terms of, you know, what you're looking for, how you're looking to do things. But I think that's important because it's going to be more customized to what your, your ultimate end state is inside
- [30:28]
Matt Sigler
of this idea of I know it when I'll see it. Right. So you know what the pattern matching thing is that you're just talking about? I know what the pattern match of a great business looks like when I see it. What's, what's the jump to understanding the translation on the AI, where you go, how your confidence, I guess, increases over time with recognizing that it's matching the appropriate amount of confidence. How do you index for that?
- [30:56]
Matt Russell
So I think one of the big breakthroughs that I have, I'll give you a specific example because I think that's maybe the best way to showcase it. When I started using Portrait, one thing that they do is they have these preloaded prompts for you. It could be the seven Powers analysis. It could be a short write up and I went in to, you know, do a short write up on A name which I owned. I was curious to see what the output was, and it showed me the prompts that it was going to use. And then I compared that to the prompts that I was using anytime I was doing deep research. And what it really crystallized for me is, oh, wait, maybe this is a me issue more than it is an AI issue. So I know we've kind of hit on the importance of prompting, but it really showcased to me, oh, wow, there's a lot more that I'm not tapping into because I'm not giving it the specificity that it needs. Oh, I always thought the model was being way too optimistic and matching what I said. Well, I can add in a line that, you know, the companies are going to be optimistic. I am looking for alternative opinions on this as well. So feel free to include a paragraph that pushes back on the analysis and what might be fluffy about it. And there's just little things there that make you kind of open up your eyes. And that was a major breakthrough for me. It's just like the power of quality prompting was something that I underestimated initially. I got better at it, and I seemingly just continuously underestimate the power of it. So I think that's just like a good example where I think many people are hearing all this talk about, you know, usability and use cases and whatnot. And there's a piece of it which is just thinking through your workflow, put it into AI, see how it could help you be more specific. And the prompt is the second piece of it, which is test out different prompts and see the different in terms of difference in terms of quality of response. And that will open up your eyes quite a bit.
- [33:09]
Matt Sigler
Experimentation is huge with this and sort of an ongoing experimentation process. The way you're explaining it is making me really, really feel that. How I love the example of actually seeing the prompts inside of the system to recognize your own. I've had several of those experiences myself. I'm curious how, where in the process, as you get deeper and deeper into this, do you re inject the experimentation phase to sort of either think in chunks or think across larger arcs to make sure you don't seed too much? I guess that's what I'm trying to say.
- [33:44]
Matt Russell
Well, there's. There's two different things. The best people that I know at using AI have very specific systems which, anytime a new model is released, they'll have their prompts which are detailed, and they'll see what this new model does in terms of output and be able to compare that to the previous model. And that's going to give them a sense of what it's doing now versus what it was previously capable of doing. I wish I did that, but I can't say that I, that I do.
- [34:14]
Matt Sigler
I just encountered how somebody actually does that this past week.
- [34:19]
Matt Russell
Yeah.
- [34:19]
Matt Sigler
And it was in comparing outputs and they were like, here's the new Claude model and here's the one from, you know, the last month versus this update. And I watch their logic. Somebody far more adept at this stuff than me critiquing each to analyze, like if this process should get upgraded to this other process. And I was like, oh, that. Yes, that's how you're supposed to do this.
- [34:44]
Matt Russell
Standardized experiments.
- [34:45]
Matt Sigler
Very controlled.
- [34:46]
Matt Russell
Yeah, controlled experiments. I think that is a perfect example of how you can do this with experimentation and I think how the best people probably do it in terms of experimentation. And you'll just hit a breaking point. Usually like in my own process, I'll get to a point where the output just, you know, the quality goes down. You know, I'm asking it follow up questions or whatever it might be, and I realize it's hit a breaking point and a lot of that has to do with my own knowledge on the subject. So that's another way to test for this is, you know, ask it things that you know about, see what the quality of responses are and you get the Gelman amnesia effect where you either start to question everything or, or you say, okay, now it's, it's much further along the curve. But I think again, going back to the beginning of the conversation, there was so much obvious value right up front with the GPT, you know, social, you know, rollout that we just had this immediate gratification, you know, sense with AI tools. And now what's happening is the people that are willing to endure the J curve of investing time, which feels wasted to get better responses is resulting in many of these people inflecting their curve massively versus many others, probably like myself, where it's a little bit more linear in terms of the, you know, not willing to endure that testing and experimentation and standardized processes to get the most out of it.
- [36:29]
Matt Sigler
Let's talk specifically about prompt writing and maybe can we do, let's do like an idea generation prompt, like what are some of the things that have to be. And the. I'm thinking about this. I want you to go explore this. What are some of the key words, variables, framings, phrasings that you use for something that's idea generation oriented yeah, if
- [36:50]
Matt Russell
we're to use a deep research LLM, starting out with telling the LLM what mindset they're in. So you are a junior analyst on the investment team at a long only fund that looks for equities, you know, undervalued with these characteristics, you know, give it a sense of its profession, how it's going to communicate all of that. The audience is an investment committee who's going to decide on this as a potential addition to the portfolio or a pass, you know, in terms of what they're looking to do. Please, you know, get deep into financials, whatever it might be. Maybe there's a KPI that you're really specific about and you think is the most important thing or maybe you're looking for that KPI that really drives the performance of that individual stock and then you work your way down. Please overweight the sources that are, you know, 10k management presentations, transcripts. Please also look for alternative sources of information. If you want to look for social media websites, whatever it might be that it has access to and you work your way down and you can get really, really detailed. I think it's good to kind of start smaller and then try it again with something even more in depth. You can keep adding to that. Then tell it what type of output you're looking for. I want this presented to me in two pages, you know, paragraph forms, bullet points at the end with the conclusions and you could be very specific about that. And you're going to get back something that is pretty darn high quality. But I think those are really, you know, what you want to experiment on. Putting your, you know, a specific mindset in, into the LLM, telling it, the audience, telling it why you're doing this and giving it a specific purpose for the output. And with the deep research model, it's again, it's agentic reasoning. So it's going to self correct and keep that north, those north stars in mind with whatever it's doing. If it hits a wall where it can't figure out the tam. Okay, let me step back, let me again think about the process and what we're ultimately looking to achieve. Maybe there's another solution here which isn't specifically tam, but something else that might be just as valuable. So that's hopefully detailed enough to give you a sense. And again, I think starting small with the prompt writing, with just telling it who they are, the audience purpose and what to look for is great. And then you could just start detailing it more and more as you go from there.
- [39:35]
Matt Sigler
I Love this reminder of the what's it do, who's it for? Aspect of saying, like, put yourself, put yourself in this position in these shoes. What about asking it? Do you ever ask or prompt questions? Do you prompt and then say, like, what else do I need to clarify? Do you do anything like that where you're asking for it to reflect on your prompts?
- [39:55]
Matt Russell
I will sometimes, prior to writing the prompt, ask it, what's the best way to lay this out? And one of the funny things, I've been trying to get up to speed with these coding agents. And I will ask just a traditional, not usually deep research but coding agent, like, what's the best way to input this into a coding agent so you can use AI to help you out with other AI and then I have yet to use it for asking how to get a better prompt. I think the, the obvious way to do it is to say, I noticed you missed this detail. What could I improve in the prompt such that this detail would be included the next time? But usually there needs to be something that you can point to as an example which will help them give you a specific answer to, you know, what you're looking for in terms of improvement. If you can't highlight what was missing, then it probably won't be able to tell you how to make it better.
- [40:59]
Matt Sigler
Right. Because then it has to assume what you meant by that. And now you're getting to a level of abstraction.
- [41:06]
Matt Russell
Yes.
- [41:07]
Matt Sigler
Yeah, it's interesting. One of the ones that I've just, I haven't outgrown it yet, so we'll see. It's like a close to a year, I feel like, of using this regularly, of asking it to, usually with a specific example in context, what I can improve, strengthen or clarify around this. And I'm looking for that kind of like those three layers. And it's weird because I feel like I'm taking something from English class. Like, this is like an editorial skill from high school, but it's argumentative logic. Like, that's all it is, right?
- [41:36]
Matt Russell
100%. And I actually think in terms of being an editor, most people will say, oh, these LLMs are so friendly and optimistic about you and will tell you what you want to hear. Just tell it. Rip this up, tear it to pieces, you know, or like, what is really missing from this piece, editorial wise? What can I clarify? What can I make more concise? What's missing? A natural transition. And it will give you great responses in that regard.
- [42:07]
Matt Sigler
So I think that by great you mean dehumanizing.
- [42:10]
Matt Russell
Well, if you tell it to be really rough on you, you'll. You'll take a step back and just, you know, get a little.
- [42:16]
Matt Sigler
Not that rough.
- [42:17]
Matt Russell
Yeah, yeah.
- [42:19]
Matt Sigler
It's also funny. I think I've. Whatever I've done, I haven't had to tell it to hold back on those things. And I've had times I've been like, oh, man, I feel bad about myself after this. I really thought I did better. And then other people, like, it only says nice things to me, like, what am I doing wrong?
- [42:36]
Matt Russell
Yeah, there's certain inputs too. Like, I set up my profile so long ago that there's certain nuances to it where it communicates in ways that are somewhat hilarious. And then I always remember, oh, yeah, I really need to go back and change that profile, particularly in chat gbt to be a little bit altered versus what I had originally thought of it as.
- [42:58]
Matt Sigler
Well, you know, they can't all think we're golden gods all the time, but it's not, it's not a bad reference point to go back with iteration on the prompt specifically, like you talked about, maybe even asking it to review a prompt before you use it, or then revisiting those prompts to sort of like restructure them in order. I'm just curious about iteration in general. Is it just part of the habit, part of the work cycle at this point where you're like, I'm going to have to consistently iterate on that first step of writing down the whole process and going through it again?
- [43:35]
Matt Russell
Yes. And I'll admit, like, this is not my strength of. I don't like to waste time of going through with a simple prompt, going through with the next version of the prompt, making sure the memory is cleared out, going through the third version of the prompt. But that whenever I do, it just opens up my eyes to how sensitive the models are to prompts, and it changes so quickly. So I do think iteration is useful. Trying different forms like, you don't have to do it. I feel this way without doing it as much as I should is all I would say, which is like, it's a meaningful impact for me. Understanding how the model works by just doing this on an irregular basis of giving it the simple prompt, adding a second paragraph, adding a third paragraph to just seeing how it tweaks the output. That is enough of a reminder for. For me to. How important it is and what leverage I can get out of it.
- [44:34]
Matt Sigler
So an investor using these tools, I'm going to assume them versus somebody who's not. There's an advantage and that grows with time. Of the people though who are adopting the tools, does it compress their abilities,
- [44:46]
Matt Russell
their skill set compress versus the others that are also adopting them?
- [44:53]
Matt Sigler
Yeah. Like if we're both using, if we're both coverage covering the energy sector and we're both using LLMs, does this actually like compress our skills because we're using these tools together, do either of us, do we end up with less edge over time in like areas of tight comparison?
- [45:10]
Matt Russell
In theory, yes. I do think because investing again is around decision making, that there's not clearly the output. We already have pretty much the same access to information. It's how we interpret the information and make decisions based on our interpretations that makes the difference. Or maybe we become aware of some insight that is not available to the rest of the market earlier, which does happen. That usually goes away fairly quickly. And I think that you could just look at quant trading, which is this in a much different form. Like we start to get into things that I don't fully appreciate. It's both machine learning, but one is oriented around language, the other is much more oriented or similar to coding around a constrained environment. You know, figure out the mathematical dynamics which result in stocks that will go up and me generating alpha and you know, make the decisions based on that. That's an area where there's not much advantage to anyone doing it versus, you know, the Rentechs and the Jane Streets and even those funds, I think struggle once they hit certain scales. So I think from a decision making standpoint, it's going to give more power to people who are good at interpreting information and then making decisions based on it. I don't think it erodes the advantage too much.
- [46:37]
Matt Sigler
If you were giving advice to say like a fun today who hadn't adopted anything yet or anybody, a DIY manager of their own capital, what would you say, like the first step is? You're, you're in charge of some portfolio. What's the first step you would use to start to adopt some of these tools?
- [46:54]
Matt Russell
I think what I've seen and part of one of the projects that I did over the summer was required me talking to a lot of institutions that are in the investment space about how they're using AI, how they're not using AI, you know, who's really far ahead, who's really behind. And one of the things that I took away was that you need to really have people experimenting at an individual level first, sharing use cases. We all do our workflows differently. This is something that you know, I was kind of aware of, but David opened my eyes to it in a different way. I remember I had colleagues that built their model before they did any other research and I thought that was crazy. But I had a different approach. Some people buy starter positions in companies before they really do much deep dive research because it makes them feel like they already have skin in the game so they have to get up to speed faster. So everyone has a different process. And I think that even exists even if a fund has a very specific mandate and process, the individuals that work across that organization probably have different approaches and you need to give them some flexibility to use the tools on their own and just get the most leverage out of that as possible and then encourage the sharing of information. So the best insights seem to come out from, you know, the Friday 30 minute stand up where the people that are really dedicated to using it share use cases and they have a free flowing conversation about presenting what they did, putting the light bulb, you know, in terms of how other people can use it, maybe getting some feedback about people who did something similar, similar and got even more leverage. So I think that's piece number one in terms of applying it from the top down, which I think is a little bit more challenging. But I know everybody's after trying to do this. I think where it's showing up the most is organizations that have very specific things that they look for in management teams or some insight that really drives their process and they have a ton of information in their archives that they can feed into a model and build their own tool that you know, really is customized to how they would want to apply it and that can feed them back information in terms of whether it's proactive research, whether it's ad hoc, on demand, that's going to be tailored to them specifically. And I think that's where like the buy versus build. I think a lot of people tried to build something on their own and most of those tools just never got used, never got adopted inside the firms. So that would be my suggestion is encourage uses, use cases that are shared openly. Don't make it a top down thing. If you are doing something top down, be very specific and be aware whether it actually makes sense to, you know, build something or if you're better off just buying the off the shelf tools.
- [49:49]
Matt Sigler
I think we're about to have probably yet another moment on the buy the off the shelf versus build it yourself with the promise of vibe coding. I feel like I've heard a lot of examples where it's going to be interesting to see what the uptake is on this stuff.
- [50:04]
Matt Russell
Yeah. And you know, it's very easy to build something that is basic. I think really turning the knobs to get it such that, you know, what you're looking for is what it puts out is in my experience, very difficult, at least in terms of what I've seen come back and the resources it requires. I can build an app, but anytime I want to take it a notch up, it starts to get pretty challenging and just requires, you know, either engineering talent or serious time and dedication to training that and really developing it in a way that I, I don't necessarily have that time or patience.
- [50:48]
Matt Sigler
Yeah. But then you get to say fun things like that's going to take a lot of tokens. I don't know.
- [50:53]
Matt Russell
Exactly. It's also expensive. That is another thing that I would mention. And you can't overlook that. Not.
- [51:01]
Matt Sigler
And this is especially good because neither of us are selling this as a service. The fear factor, the FOMO of falling behind, like if somebody's just not using this or isn't adapting, do you see that as an existential threat to falling behind if you're not diving into any of this at all specific to asset managers or people in the investment space?
- [51:23]
Matt Russell
Eventually, yes. Like, I don't think that any fear that exists today is warranted beyond, you know, that fear acting as a catalyst to get them to adopt it. All I would say is if, if you're not using these tools in some way, you're probably a lot less productive than you could be. And what, what's guiding this also is that the most advanced investors, in terms of how they're using it, I still don't think it solved this really obvious alpha generating solution. Now I've been very impressed in terms of some of the insights that people are now able to pull out based on tools that they've set up, different ways that they're using the AI to interpret or extrapolate information. And most of this is just coming out as an extension of what they might have otherwise done, but they're able to do it faster with quicker iterations and because they have more free time, because they're not spending so much time on other things. So I would answer that in two ways. I do think over time it's going to be like using a Bloomberg or using a calculator or the difference between, you know, writing your model on graph paper versus putting it into Excel and you know, versus the tools that we have today. So I think that's piece Number one is you would look at somebody incredibly silly if they're not doing that. Unless it's somebody like Buffett where it's like, oh, napkin math. Would he need to use AI? Probably not, but I bet he would because of, you know, all the value that could come from it. And then piece number two is the reason why I don't think it's like a huge fear today is just because I don't think that there's this alpha opportunity that is linear to the usage. I think it's indirect and it's because of the productivity and efficiency that comes inside the system.
- [53:22]
Matt Sigler
Let's say you're like a mid career analyst or somebody where you have a, you've developed a specialty, you're far enough in your career that you have a specialty. Is this the time, like, is it better to be like a generalist and know how to use these tools? Or it's still a good thing to have like an energy or a utility or a specialist in like a certain sector or area that direct domain knowledge.
- [53:48]
Matt Russell
It's a great question. My gut response is that this is going to make generalists that have a good sense of investment decision making much more valuable because they can get up to speed faster, they can digest information faster, they can gather information much faster than previously. And those were barriers before. Now it doesn't completely close the gap to your dedicated energy analyst, but I think it shrinks the gap quite a bit that you can cover much more ground. So I do think it's somewhat of an advantage to, or I do think it's an advantage to the generalist or those that run lean, like those teams that are already lean will have an advantage because this will only make them more efficient. So it's kind of something that already existed in terms of where their edge might have been. And it, and it theoretically extends that edge.
- [54:48]
Matt Sigler
And in a way it kind of, it might work more in like reducing potential blind spots. Right. It might be helping with lessening errors versus opening up.
- [54:58]
Matt Russell
Yes. And, and goes without saying, you're never going to be perfect with that. So I don't think, you know, a specialist could take that, you know, and push back on it. It doesn't completely go away. There's always going to be value in being a specialist and really understanding how these things work. But I think it shrinks the gap quite a bit.
- [55:21]
Matt Sigler
Yeah, I think of all the stories, it's like the hedge fund analyst stories where they're like, I got this thing exactly right for entirely the wrong set of reasons.
- [55:31]
Matt Russell
Yes.
- [55:32]
Matt Sigler
I think of how many smart people I've heard utter a version of that sentence, like, well, we were there. The penny for the reason you think?
- [55:40]
Matt Russell
Yeah, yeah, yep.
- [55:43]
Matt Sigler
How do you think, how does this change the pursuit of alpha, of outperformance, of this kind of stuff, do you think? Does this just further compress, like the inability of active fund managers to outperform? Does this, this have any impact? Is this just meaningless fluff on the top of this conversation?
- [56:01]
Matt Russell
Well, it's interesting because there's the impact of AI, the technology and all of the capital that is going into the development of this technology and then there's the actual, you know, impact of using it within the investment process. So it's, it's kind of a double edged sword in terms of the impacts that it has. I do think that we're seeing these drastic swings, this massive capital cycle and yeah, it can massively change things, but I don't know how anybody has a real like perfect vision of what that will look like. We've already seen this massive shift towards passive quant, all of that. I don't think stock picking will ever go away. I don't think dealmaking will ever go away. So the things that, you know, have maybe shrunk but still represent a large percentage, at least in gross terms of the world and, you know, a potential still exists. So I'm, I'm only bullish that, you know, it makes you more efficient. And if you believe in your ability to come to different conclusions versus, you know, the market, which you have to have some bold confidence in being able to say that. But if you do, then this should make you better at your job.
- [57:21]
Sponsor/Ad Voice
New Year, new me. Cute. But how about New Year, new money? With Experian, you can actually take control of your finances. Check your FICO score, find ways to save and get matched with credit card offers, giving you time to power through those New Year's goals. You know you're going to crush. Start the year off right. Download the Experian app based on FICO scoring model offers an approval not guaranteed. Eligibility requirements and terms apply subject to credit check which may impact your credit scores. Offers not available in all states. See experian.com for details.
- [57:48]
Matt Russell
Experian
- [57:52]
Matt Sigler
Five years from now, what do you think we're looking back on and saying this is not to bait you into commenting on the Citrini piece. This is to say at some point in the future, not too long from now. Looking back, what do you think the most obvious themes of how investors are using AI? Not Asking for a market prediction how people are using it.
- [58:11]
Matt Russell
I just think the customization of these tools to individuals is going to be massive and I think it's really going to open up our eyes to, you know, everything is a standardized thing like your Google search, it's a standardized algorithm. You know, you go to a website, you want to look at different shirts, it's a standardized thing. And you get some model that might look nothing like you. All of these things exist out there. You know, you're getting this model sent back to you in a standard template that whatever, you know, the off the shelf company decided or the sell side company decided, that's how it's going to be spit out. And I think everything is just going to be extremely customized to however we want it to be. And that is probably doesn't feel like that big of a deal. But I think it's really going to open up our eyes to how much better it can be when things are individualized and made for you. And I think, you know, agents and agentic, again agents is this blurry thing and it's just like an extension of LLMs. But agentic AI is something that will have the capacity to do a lot of things, present a lot of things, be created for you in a way that can serve you in an almost human like assistant form. And I think it's starting to click, but it's really going to show up a lot more with what exists in the work processes today. And just as the models get better, but then also new tools that are created for you and your ability to customize those tools for you, which will start to feel really amazing. And that's when I think your question about does the alpha potential go away? All of a sudden you're going to have information at your fingertips incredibly fast and it's going to be about, you know, looking for what information you know you need to look for and having that sense. And does the AI ever get better at telling you which information you should be looking for? That's a question I think is way further down the line than five years. But that's what I would say, way
- [60:26]
Matt Sigler
further down the line than five years. And I think like remind people of this all the time. I think it's a very valuable complaint that I, that I had for a very long time inside of large financial institutions which was basically like why is the fax machine still here? And we might get all this amazing stuff. You might be able to imagine this, attach a PDF to an email, but somebody's still going to ask you to fax something, and it's going to be a hard process.
- [60:54]
Matt Russell
There's a lot. There's physical cash. While we have credit cards, there's credit cards. While we have Apple Pay. There is all types of things that are just in the system and they're there for a reason. So I think adoption in general is going to take time with large enterprises. Even though I've been impressed at what's happened, that just means if you do adopt these things, you'll probably have an efficiency advantage so you can, you know, enjoy your happy hours, you know, instead of being stuck in the office for certain things. So you should, you know, take advantage of that. And, you know, I think there's another thing right now in a lot of organizations where, especially in the investment world, where if you go into, you know, out on the west coast in the tech scene, if you're not using AI, it's like, get out of here. But in some of these organizations out here, it's like if you're using AI, that's viewed with a negative lens. So you kind of have to hide using the AI. And I do think that's a skill. Like some people do not hide it and it becomes a mark. It's like product placement in things, sponsored content, whatever it might be. But those are two things that I, I think about quite a bit. Like there's an advantage to people who are using it really effectively, not showcasing that it's all coming from AI and, and yes, their input is still in there, but it's making a difference.
- [62:13]
Matt Sigler
It's definitely making a difference. What are you personally most excited for? What's. What's getting you out of bed in the morning where you're like, oh, got that refresh in my data, can't wait to get in there.
- [62:24]
Matt Russell
I. The coding agents are things that I've been, you know, playing around with. A great example of something where I always told myself, you know, I'll teach myself how to code one day. I don't need to take a course or whatever it is I never taught myself.
- [62:36]
Matt Sigler
I've been living that life for 30 years. Do I really have to give it up now?
- [62:39]
Sponsor/Ad Voice
Yeah.
- [62:39]
Matt Russell
And cursor opened up my eyes to, oh, wow, this just got insanely easier. And then I, you know, moved to Claude code. Just incredible experience. Been trying Codex with Chat GPT, hit some humps, but I'm hitting my ceilings. But it's just making me excited that I can create things that previously I wasn't able to create or didn't have the stubbornness to do. And the iteration cycles happen so fast where if I had either taught myself or was working with, you know, a contractor on it, I would have given up a million times. But it's just so fast that I keep coming back to it. And we'll see. We'll see if I can ever, you know, make something out of it.
- [63:23]
Matt Sigler
I look forward to you and I creating our first Angelfire website together.
- [63:28]
Matt Russell
I wish those archives were still around. Let me just say that I got a few golden ones. I'll have to go in the way.
- [63:34]
Matt Sigler
You and me both. Yeah, one or some bangers and probably still not share them with the world.
- [63:41]
Matt Russell
Definitely not.
- [63:42]
Matt Sigler
But I definitely have a few that I at least would love to recreate. Matt, this is. This is awesome. This is a topic that I feel like is just going to keep coming and if we don't stop to slow down and talk through this, which is why I want to capture this in public. Where should people follow you? Where can they see some of your work on this? We're going to put the link to that Business Breakdowns in the comments here.
- [64:01]
Matt Russell
Where else on Twitter? Russell, Matt. Last name, first name matt.russell.com an infrequent writing spot for me that I do from time to time. Those are the two spots that, you know, I try to stay communicating with the world and then Business Breakdowns Podcast is another spot.
- [64:20]
Matt Sigler
Well, we like the sound of that. We like the website not being full of AI writing. Way to keep it real. Matt, thank you so much for joining us. This is Excess Returns. Like subscribe, comment, all the things below. We are out.
- [64:32]
Matt Russell
Thank you for tuning in to this episode. If you found this discussion interesting and valuable, please subscribe on your favorite audio platform or on YouTube. You can also follow all the podcasts in the Excess returns network@excessreturnspod.com if you have any feedback or questions, you can contact us@excessreturnspodmail.com no information on this podcast
- [64:52]
Sponsor/Ad Voice
should be construed as investment advice. Securities discussed in the podcast may be
- [64:57]
Matt Russell
holdings of the firms of the hosts.
- [64:58]
Matt Sigler
Close your eyes. Exhale. Feel your body relax and let go
- [65:03]
Matt Russell
of whatever you're carrying to.
- [65:06]
Sponsor/Ad Voice
Well, I'm letting go of the worry that I wouldn't get my new contacts in time for this class. I got them delivered free from 1-800-contacts. Oh my gosh, they're so fast.
- [65:14]
Matt Russell
And breathe.
- [65:15]
Sponsor/Ad Voice
Oh, sorry. I almost couldn't breathe when I saw the discount they gave me on my first order. Oh, sorry. Namaste.
- [65:23]
Matt Russell
Visit 1-800-contacts.com today to save on your first order. 1-800-contacts lifelock. How can I help?
- [65:30]
Sponsor/Ad Voice
The IRS said I filed my return, but I haven't.
- [65:33]
Matt Sigler
One in four tax paying Americans has paid the price of identity fraud.
- [65:37]
Sponsor/Ad Voice
What do I do?
- [65:37]
Matt Russell
My refund though. I'm freaking out.
- [65:39]
Matt Sigler
Don't worry.
- [65:40]
Matt Russell
I can fix this.
- [65:41]
Matt Sigler
LifeLock fixes identity theft guaranteed and gets your money back with up to $3 million in coverage.
- [65:46]
Sponsor/Ad Voice
I'm so relieved.
- [65:47]
Matt Russell
No problem. I'll be with you every step of the way.
- [65:49]
Matt Sigler
One in four was a fraud paying American. Not anymore. Save up to 40% your first year. Visit lifelock.com podcast Terms apply for their clients.