C (3:54)

As a quick follow up on that is you're talking a little just for the guests that may not be familiar with Mozilla and the kind of the context of Thunderbird within Mozilla. Could you talk a little bit about that for a moment just to, to kind of set the stage?

D (4:07)

Yes, I will do my best. It is a little convoluted, as Wikipedia can tell you. So Thunderbird. All right. You can think of Thunderbird? Well, first of all, for those who don't know, for the kids out there who are like, what is email? Thunderbird is a desktop email client and for a certain generation those three words strung together means nothing. But it is a 20 plus year old open source project which originated at the company now known as Mozilla. If you're familiar with the Firefox browser that is made by our sister company, the Mozilla Corporation, and Thunderbird and the Mozilla Corporation all fall within, I guess you could say the guidance of the Mozilla foundation, which is a nonprofit and as of, oh, I guess it was 5, 5ish years ago, maybe a little bit longer than that. Mozilla was, nobody was maintaining Thunderbird. So this application that was not Firefox had, you know, they didn't want to deal with the maintenance that was, you know, engineers, engineers need to maintain this almost 20 year old C code base. And that was for, you know, management reasons are like, okay, we could just hand this to the community or we could just shut it down. And the director of product here at who's now the director of product, Ryan said, wait a minute, listen, give, give me a shot. Let me, let me try something out. Let me see if I can turn this into something more than it is now. And they did whatever legal things and paperwork was necessary to spin it off into its own entity. We are known as Thunderbird or the Thunderbird Project. But if you look us up on the Internet, we're officially mzla, which sometimes I feel like it's one of those acronyms that doesn't really mean anything, but that's what it says on my resume. So that's, that's Thunderbird in a nutshell.

B (6:08)

Now. Yeah, you mentioned this kind of history, this sort of desktop email application history. Just for context. I'm sure there's many people listening that do remember those sorts of days, but others that, that might have just always used Gmail in a browser or something like that. But there's a whole generation of us that use something, whether it's Outlook a lot of people have used. I remember installing Ubuntu for the first time earlier in my career and using Thunderbird in that context on my computer. Now you mentioned there's kind of maybe a, a different, a more vision for this now, any context you could provide there in terms of, you know, the way people use email now versus those days when Thunderbird was being used as that desktop application. What is kind of the, I guess the focus and transition, if you will, or at least some of the things that are being thought about.

D (7:14)

Yeah, I'll start with sort of the, the ethos like why, why is Thunderbird still around? There are still people who use email. It works perfectly well. But unlike using say a webmail based provider, these folks are not interested in having ads sort of injected for them and maybe they, they want to be able to opt in to AI. They don't want an AI just like ever present and reading their emails. So with Thunderbird it is, it's free and open source, no ads ever, forever. And your email is your own. It gets downloaded to your computer. So if you lost Internet access, you can still read your email. If you lost power and your laptop is open to Thunderbird, you've still got it right. All those emails that you downloaded, they're right there. So I think that the way that email is being used, it's different. It's different. Now one of the great things about Thunderbird back in the early days, um, and it's still true today, is it's very easy to manage multiple email accounts. Um, that was, you know, I remember a time when like, oh, you have more than one email address. Amazing. Um, you must be really important. And now you can just sign up for email addresses, dozens of them if you want, and you know, have them each dedicated to a different purpose. You can still do that With Thunderbird, however, one of the things that people are encountering now, as we all are, is a certain amount of information overload. Right? You're subscribed to so many newsletters and mailing lists and you're working on some collaborating on some side project and then you've got your main work email. How do you read all that email? How do you deal with all of it? So those are the sorts of things that we're thinking about now is how do we empower the user to better deal with just this huge influx of information that they're getting every day, every hour.

B (9:24)

So first off, thank you for validating my multiple email use. Every once in a while my wife gives me a hard time because I generate a new email every once in a while when I get frustrated with my email feed or have a specific purpose. And it seems like no one knows, no one ever knows which email to email me at. Which maybe is a strategy in and of itself, I'm not sure. So thanks for validating that. But yeah, I know that there's a good amount of kind of AI intersection with email, whether that be from like the web email side. So in Gmail or other things with, with Gemini or email clients that maybe are specifically geared towards AI features like a superhuman or these sorts of things. Not asking you to comment on every single one of those, but maybe in general, like how from your perspective have you viewed this gradual integration with AI and what sort of, I guess, categories could people have in their mind of the kinds of ways that AI is being applied within email and some of those trade offs that maybe you're making when you're using those features that of course I know we'll talk about privacy in some of what we talk about with Thunderbird, but maybe just from a perspective would be curious on how you categorize AI in email. Could mean a lot of things. Could you help people understand maybe some of that landscape?

D (11:06)

Sure, yeah. I think that the two main ways that I see, I mean in my own Gmail account, because I will use a device that doesn't have Thunderbird on it because I'm a longtime distro hopper and maybe I'm using a distro that doesn't have Thunderbird prepackaged and I'm just trying things out. I have found that the automatic summarization is a thing that Gmail's like, hey, Gemini can do this for you or all of the autocomplete that it tries to helpfully offer me. I feel like it's a little Creepy sometimes, especially depending on what the email is. Like if I'm talking to my doctor over email and it's like, clearly this LLM has read this very private information. I'm like, oh, how do I turn this off? But I do understand that for a lot of people those two features, they are time savers. You have this way of compressing more human time into your day by offloading it to an LLM. So that's great. It's a time saver and it's great that people have access to that. Some things that you lose out on that you're gaining time. But what you're trading are things like tone. The tone is for some summarization models, it will kind of strip out all the tone. Or if it's bulk summarizing multiple emails, that email from your mom doesn't really sound like your mom. It is literally just like your mother's coming to town this weekend. So there's maybe dehumanization of email is kind of the wrong term or maybe a little extreme, but that gets essentially normalized to the tone of the LLM. And yeah, to your point, the, the privacy aspect, that's, that's kind of the, the big one for us here at Thunderbird. We're very privacy respecting, privacy preserving because that, a lot of our users choose to use Thunderbird because of that. They want to manage and own their own email. They don't want, they don't want their personal emails harvested for marketing purposes or for training data.

E (13:40)

Well, friends, when you're building and shipping AI products at scale, there's one constant complexity. Yes, you're wrangling models, data pipelines, deployment, infrastructure, and then someone says, now let's turn this into a business. Cue the chaos. That's where Shopify steps in. Whether you're spinning up a storefront for your AI powered app or launching a brand around the tools you built, Shopify is the commerce platform trusted by millions of businesses and 10% of all US E commerce. From names like Mattel Gymshark to founders just like you, with literally hundreds of ready to use templates, powerful built in marketing tools and AI that writes product descriptions for you, headlines, even polishes your product photography. Shopify doesn't just get you selling, it makes you look good doing it. And we love it. We use it here at Changelog. Check us out. Merch.changelog.com that's our storefront and it handles the heavy lifting to payments, inventory, returns, shipping, even global logistics. It's like having an ops team built in into your stack to help you sell. So if you're ready to sell, you are ready for Shopify. Sign up now for your $1 per month trial and start selling today at shopify.compracticalai Again, that is shopify.compracticalai.

B (15:08)

Well, Chris, we were starting to get into a little bit of, I guess, the intersection of the ethos of Thunderbird with these sorts of AI features. Now, could you help us understand, I guess, like we talked about autocomplete, we talked about summarization, for example, there are various mechanisms by which these features can be implemented in an email client or an application or web application, whatever that is, in terms of the actual AI model, where it sits, how the data flows, what the model is trained on. Maybe could you help us understand that piece? Like, what are the buffet of options available to us in terms of how we might like the integration point in the flow of data?

D (16:00)

Sure, yeah. We have discussed at length different ways that we could approach this. So let me begin by saying that this experimental work in bringing an AI assistant to Thunderbird, this is not baked into Thunderbird. We're not going to turn it on for users. It's not going to be automatic or anything like that. Instead, what we've done is we have built it, as I'm going to call it, a companion for right now. We'll put a pin in that. We'll return to that because there's a lot of decisions we had to make because of that approach. Now, our options that were available to us, we could just do add model inference to Thunderbird itself. This is, in my opinion. Yeah, but now this is us turning it on for all users. It's like, just add the model in there and allow us to do inference locally. It's private. Right. And that's great. However, we're not so invested in this idea that we want to put that on the desktop team's roadmap. When I say the desktop team, just a little plug for the mobile team at Thunderbird, you can grab it for Android now. Works really well. IOS is coming soon. But as far as the desktop client goes, they already had their roadmap and we just kind of want to run our AI experiment parallel to that. Okay, so that's one option. A second option is like, well, could we. Could we run inference in a separate application, also locally? Yes, we can, and we've kind of poked around with that. But we don't necessarily, if we want to roll this out, we don't necessarily want to require users to download a second application. Okay. So, well, how do we split the difference here? All right, so what if we called out to an API, like one of the cloud providers for, you know, any of the, the models that would be good at helping you out with email, the typical tasks of summarization or reply generation. So we decided that could be a thing because it doesn't require installing anything heavyweight and additional. But that brings up a separate problem of like, where are you sending the email data to? And I'm happy I will be talking more at length about that.

B (18:31)

Yeah, yeah, I think in all my discussions in my day job work, that's often what it comes down to is we would love your AI features. Where is the data going? So I definitely understand that and I assume that there's definitely this tension of doing things locally and putting that on. You said the desktop roadmap, but also there would potentially be kind of either limitations in terms of the kind of model you could use or even just if you could use a model that worked well, but it might just destroy all of the battery of the device on which it's running, or that sort of thing. Were those things also part of that conversation?

D (19:25)

They definitely were. Especially regarding the second application that just ran a model inference process in the background. I don't know about you, but my laptop, my work laptop, not super fancy, it generates a few tokens per second, which is, it's just not fast enough for this, I mean literally for me to work on the thing that I'm working on. So that was a big concern that and a lot of our users are on Linux and they're running Linux because maybe they're continuing to use their perfectly good hardware from 10 years ago, which most certainly cannot do any sort of local model, anything. But yeah, laptop users, I love my battery life. I don't want to completely destroy it by trying to summarize a batch of today's emails.

B (20:21)

Yeah, yeah, that makes sense. So you mentioned this idea of using a model that is behind a remote API. There's obviously a selection, we're kind of narrowing in. There's a sort of selection of ways that that could happen. And there's like various approaches around maintaining privacy there from just using an API that explicitly doesn't store certain data, at least according to their terms, or doesn't train on your data, at least according to their terms. There's also kind of, you know, I know people are exploring kind of homomorphic encryption and all of these things to like keep, keep data private. And then there's sort of end to end encryption and there's, there's all sorts of ways that you could think about privacy in that context. What were the main, I guess the main pillars of what were important for you all to consider was that where data is sort at rest, is it the openness of the models or whether you were hosting those or a third party was hosting those? Was it? Yeah. What were kind of the main topics that came up once you kind of dipped into that remote inference side of things?

D (21:46)

That is a great question. And I'm going to try to condense the story down.

B (21:51)

Okay, great.

D (21:52)

It'll be kind of like the Hobbit and then the three parts of the rest of the story.

B (21:58)

Sure.

D (21:59)

Wow. You should just totally revoke my nerd card now for not coming up off.

B (22:05)

The top of the Fellowship of the Ring.

D (22:07)

That's the one.

B (22:07)

Exactly.

D (22:08)

I just lost half of your listeners.

B (22:10)

The two towers. Return of the King. Yeah, you got it. I, at least I might not know about the cool encryption stuff that you're about to talk about or whatever that is, but at least I have that one.

D (22:26)

Nice. Well, our powers combined.

B (22:29)

The powers combined. It takes a community. Yes, yes.

D (22:34)

So let me start this little story off with the very earliest experiment with this was as a Thunderbird add on. What is a Thunderbird add on? A Thunderbird add on you're familiar with and your listeners are probably familiar with browser extensions. Fun fact, Thunderbird under the hood uses the Gecko engine from Firefox. So we have access to APIs that can make it possible so that something almost exactly like a browser extension can reside within Thunderbird. If you're not familiar with browser extension development, it's basically like HTML, CSS and JavaScript. So we started by writing something that was, it was the most 1990s looking webpage that was just sort of like jammed into an add on and just displayed in a new tab in Thunderbird. And it was, I mean, you know, it looked like an engineer built it and that's totally fine. But yeah, that's when we started with calling. Like we started off with OpenAI's API and just handing off a number of emails to like, I think it was ChatGPT4. And that did a, that did an okay job, but that was like, okay, it does work. How do we build this out? And then we started trying to get better results with some prompts, tuning and whatnot. And then as we started, you know, trying to use it with more people within Thunderbird, we found out like these people, their emails are sensitive. Like we don't what do we, we need to do something about this. So we started shopping around for some sort of cloud based provider that could give us a guarantee of yes, we do not store your data. No, we're not using it for training. We were in contact with a couple of different companies, some of whom just sent us to one of the pages on their website which told us nothing useful, couldn't give us a good guarantee. That's when we started talking to the folks at Flower Labs. And I know that you have had several guests from Flower on the show. I just want to say that they are so terrific to work with. They took care of, I mean really, they took care of like all of our needs. They moved things around on their own development roadmap and gave us early access to things like end to end encryption and access to their newest product, Flower Intelligence, which it is. So for the listeners who hadn't heard of Flower or listened to previous episodes, they're known for their federated learning SDK, right, that build software that does learning on individual nodes and then shares the learnings with centralized server. All very cool stuff. We didn't need that though. We needed private API or rather API access to a private LLM. What we got in addition to that is we got a nice SDK in TypeScript and we also got. So we got the end to end encryption and. Oh, right, they found a model for us and then they did some post training on email summarization. They built an eval system so they could fine tune. They help produce prompts, you know, through the eval system. I mean they just, they've been incredible.

C (26:28)

I'm just curious as you, as you talk about this and especially having had multiple folks from Flower on the show in the past, you're talking a little bit about kind of how you got into the collaboration. But like how did you, how did Flower come into the picture to begin with for you guys? You know, how was that connection made and how as you were looking at that connection potential, how did you know that that was a good fit for this new strategy that you've been laying out?

D (26:57)

I have the most boring yet magical answer to that question. It just kind of fell into our laps because Mozilla, all right, remember I was talking about the, the nonprofit Mozilla foundation under which Thunderbird sits. They're an investor in Flower. And so Mark Sermon connected Ryan, my boss's boss's boss, with Daniel from Flower, and I just ended up on a zoom with him one day. Ryan introduced us and said, all right, take it away and that's when the collaboration began. So that's the thing, I guess having a human face to go with the company made me feel good about those guarantees of like, no, we're literally not in the business of harvesting data. Like, we, we're going to set up this infrastructure and we literally can't help you debug your prompt because it's encrypted now. So that was the fact that they, they've been. So they were so helpful at every step. Except when I sent some bad data. Like the fact that they were like, we really want to help you, but we built it in a way that we can't, we can't see it, the data. So that was, it was a fortuitous connection, thanks to the reach of the Mozilla Foundation.

B (28:13)

Well, Chris, you started unveiling some of what made this particular route of experimentation useful for you all. Could you help us understand, maybe just at a slightly deeper level? I could, let's say, spin up a model in vllm, in a VM, on GCP or wherever I host things, and then connect to that over an API. What kind of makes the hosting of the private model within the flower system because like you say, it's not federated, but there is still kind of more there, as you mentioned. Of course, there's the post training that you talked about, but as far as the inference side, could you help us understand that a little bit more?

D (29:04)

Yes. So as I mentioned, they've taken care of all of that, which is. That is the big benefit. They've really been a great technical partner while we conducted these experiments. And from the privacy aspect, my coworkers no longer have to prune their inbox and remove anything sensitive before trying out what at this point we have dubbed Thunderbird Assist. It is your personal executive assistant within Thunderbird. That was the idea. Anyway, thanks to the guarantees made by Flower, we were then able to try out. I mean, we didn't land on the current model immediately. We tried different Berts, different Barts, Roberta, all the various summarization models. And then one day the summaries got way better and I said, what did you do? What is this magic? They switched to one of the llamas from Meta that was trained for conversation and it worked better for email content. So at that point we stopped thinking about the prompt because they had squared that away. And prior to that they found a model that could do the task very well. So then that got to the part where we're like, okay, so it's really great at Summarization and reply generation, the third feature that we worked on, this is the biggest thing we aimed for because when you think about like, oh, hey, you're doing email summarization and reply generation, that's great. That's basically the hello world of LLMs. Summarize this text. So we started working on something that was. Or we had been working on this feature that was. It did not work well. We refer to it as the Daily Brief and it was intended to be an executive summary of your recent emails. This is when I learned the definition of over prompting. What I would do is I would take however many emails arrived in the last 24 to 48 hours, ship it off to the model, and then ask it to do, oh, can you find the most important messages, extract all of the highlights and any action items and return those back to me, grouped in this particular way with links back to the original emails. I mean, the garbage that I got back sometimes was epic. So I then learned that, okay, what I need to do is I need to split this up into multiple tasks, multiple requests, right? Let's send a batch and only ask for importance. Some of your listeners are like, okay, that's highly subjective and I will return to that momentarily. But when we provided the emails, we also provided the unique message IDs from Thunderbird. So that way I could then use that as an index, grab the original emails again and send off a second request which was like, okay, so for these, I want you to find the action items and then take the same batch. Now ask for highlights and crucial information. The formatting task never worked well, so I have lots of feelings about that. So we'll put a pin in that. The formatting never turned out because that was what I wanted as an application developer. The LLM was not capable of. It was, it's a statistical model. And I guess it's thinking that maybe they want the header bolded sometimes and maybe they don't. I'm like, LLM, do what I want. And it's like, who knows? I'll roll the dice. Maybe I'll give it to you the way that you want. And so that was an important lesson, was like, okay, so in this currently, you really need to be very careful, specific and constrained in what you ask the model for. Then the problem turned into, okay, so how long does it take to make these subsequent requests? What can we parallelize? How do we do that effectively without burning up all of Flower's compute in their infrastructure? We started looking for ways to optimize that and we Switch to a local Bayesian classifier, we'll take the first of several tasks and we'll just do that locally. Instead of asking an LLM to very subjectively decide what sounds important or what the cosine similarity algorithm tells them is important, we'll do that locally. We'll let the user use the Thunderbird feature of tagging emails as priority one, priority two, whatever. So for our experimentation, we had each user tag a handful of messages as highest importance and tag a handful as least important. And then the local Bayesian classifier that we just included as a JavaScript library in the Thunderbird add on works very quickly, even for lots and lots and lots of messages. And so, okay, task number one taken off the plate of the LLM and so now we just have it do the rest of the tasks and likewise the formatting task, we just handle that ourselves. A quick note about formatting. For a time when we were using one of the other third party cloud providers, we found that you could provide them a JSON schema that the model would conform to when giving you the response. That was a magical time for me as an application developer because it's like, oh yeah, give me the JSON, I will just put it through my framework and it's just going to render the things beautifully. Like, look at my CSS applied so perfectly to this. This is amazing. When we, as we were model hopping and provider hopping, you know, that kind of went away and we haven't returned back to it because at some point we realized that, okay, so learning our lesson from before about splitting up tasks, we realized that we need to take a different approach for the daily brief. And this is when I got it in my head that like, okay, so the future of this feature is not to just keep on sequentially prompting the same language model for like, hey, now do this, now do this. Instead, I think of like, you know, you go into a professional chef's kitchen and you don't see like this one giant tool that can slice, dice and microwave and air fry. On top of that you see lots of little dedicated tools that like in expert hands, it. It does, it does that one thing and it's going to do it the best. So one of the things that we've got that I have written down in my Mad Science notebook is to explore like, well, what if we could dedicate some small models to specific tasks and then coordinate them in some sort of much more deterministic way? So the daily brief currently, all that to say, the daily brief currently has been kind of sidelined. And we're like, okay, so shipping assist means more or less summarization and reply generation and daily brief. We still need to work on that because again, the approach needs to be more granular and more deterministic.

C (36:59)

I'm kind of curious as you've taken us through that process. One of the things on my mind is how different, kind of, for lack of a better word, audiences within your customer base, different profiles, how these different approaches that you've taken as you guys have devised the strategy forward on Thunderbird, where are you seeing more uptake? Where are you seeing, you know, people? Like, you know, we, I think in. Early in the conversation we talked about younger generation who may not have grown up with email like we did. And then on the, on the far side of that, you have kind of the corporate world and stuff like that. And with a, with a, you know, a certain segment of your, of your user base in different aspects. I'm just curious how that may have. How people are receiving this in those different capacities, given the fact that you have different interests and stuff.

D (37:52)

Yeah, yeah. Well, the short answer is that our users are fairly homogenous at this point. Our users of Thunderbird Assist are very homogenous. They are all Thunderbird employees. Because it is. Yeah, it does, it does. And even though this work is. It is open Source, it's on GitHub right now. We haven't released it for general use because again, Flower had been tuning the models and making changes to their infrastructure so they weren't ready to receive a lot of users from all over the world. So within Thunderbird, there are even different needs. Some people use the individual email summary feature. Okay, so let me back up. There are three features that are available in Thunderbird Assist, the first of which individual email summarization. And if you aim it at like a quoted thread, you could call it a thread summarizer. There's email reply generation, and then the third one is the daily brief. Now for the different kinds of users fall into two camps. There are the. I would like this long thread from the Thunderbird mailing list. I need that summarized because, wow, that is too long. The other kind of user is the one who gets way too many emails and needs an executive assistant. And that was where, as I just mentioned, it's like, okay, so that feature is just not going to work very well given the approach that we've taken. But based on what some of the users have requested in response to using Thunderbird Assist, that's given us some ideas of like, well, what we should be focusing on is we need things like semantic search or because it is a Thunderbird add on and has access to more than email. Like it has access to your calendar account that's in Thunderbird. Thunderbird also does task management and it even pulls in RSS feeds, which rss. It's coming back so strong. I love rss. I think that this idea of like, okay, so if we could correlate between these different pools of information, that could be extremely useful to some users. Which brings us back to the whole, like, okay, well, you need small dedicated models for each kind of data because they're going to be formatted very differently.

B (40:38)

Well, Chris, I have sort of two. Well, an observation and then maybe a clarification. So number one, I love how you described this progression from kind of the one tool to accomplish every task, which is often how people do think about using these models, down to splitting this out into maybe it could be different models, it could just be different applications of the same model, but that are segmented or these sorts of things. This is so often what I recommend to people. It's kind of like when you have a junior developer and they come to you and they're like, I wrote all the functionality, it's all in this one function, right? Thousands of lines of code. And you're like, okay, we need to split this up, right? It's sort of in this AI world, there's that need for that splitting up and of course it makes things more testable and all of that as well. So thanks for highlighting that. I think that's a really, really practical and good point. I think the clarification. Just want to make sure that people picked up on you kind of referenced some of this work with Flower and we talked about that remote inference, if I'm understanding, right, because you're running a local application, the data that flows to that remote inference is encrypted on the device, so it's encrypted in transit. And then if I'm understanding Flower's implementation, you can correct me if I'm wrong. That would only be decrypted sort of in a confidential enclave in the inference infrastructure. Right? So that's when you say, like, even Flower, even if this is running in their infrastructure, they would not be able to tell you what a prompt is. Did I pick up on that somewhat in the right vein?

D (42:29)

Absolutely, that is totally correct. It is. For any web developers or anybody who has had to write software that interacts with an API, you're probably communicating over something called HTTPs, which is sort of like, okay, that is the baseline amount of encryption that we want. It's going to encrypt the traffic between your browser, the client and then the server. They take it a step further. I would call it a three part process for making sure that your data is protected. First off, let's say you're logged in, you log into your Thunderbird account, which we created specifically for Assist and some other services which I will talk about a bit later. Then you are issued an API authentication token from Flower itself. Right. You're logged in. You're now going to talk to Flower. Flower's like, yep, you logged in through Thunderbird. Cool. Here's your authentication token. Use this now to exchange public keys with yet another server. And that server does nothing besides run the, you know, run the language model. And at that point, as you, as you observed that any traffic between your client between Thunderbird Assist specifically and then the machine running the model, like it's all encrypted in between. So you have, yeah, you're double protected, I guess, HTTPs + the public key encryption between you and the, the model server.

B (44:14)

That's great. Yeah. And I think this is a great way to maybe expand people's thought process around what's possible with privacy and LLMs and how that can be split up between like where the LLM is running, whether that's local or not, or both. So yeah, appreciate, appreciate you going into a few of those details. I think it's really helpful as we, as we do get closer to the, to the end here. I would love to maybe just kind of ask you to close us out by thinking about the future. Now that you've run these experiments, you've kind of gone through this process. I love how we kind of went through this kind of story of how this developed. Now that you've gone through that process, as you look towards the future, what excites you about where things are at now and where they're headed in this project or maybe in terms of like the wider ecosystem that you're now a part of, using this tooling around kind of remote confidential inference and that sort of thing.

D (45:23)

Yeah, there are a lot of exciting directions that we could take this work. Again, this was sort of an initial experiment, but we are planning on shipping this with what we're calling Thunderbird Pro, which is a suite of services. Like my other web developer teammates, they're working on other things like a scheduling application, a web based scheduling application. There's an end to end encrypted file Sending application. And there's Thundermail. I'm just going to say that again. Thundermail, which is our very own email service. Okay, so one of the things that could be very interesting and perhaps even take advantage of federated learning, thanks Flower. Is if you could treat the server, the email server, or you know, another machine that is co located with the server as another client, right. That has access to your encrypted email that's on Thunder Mail that while you're asleep or while you are disconnected from the Internet, it could be creating embeddings or doing some other inference based on your email data and then transmitting the learnings to your local machine. So that. Imagine if you could do semantic search without having to generate the embeddings on your laptop. And you can do it in an offline way because as far as you're concerned, the embeddings are effectively pre generated and downloaded along with the messages themselves. The trick there of course is doing it in a way that will satisfy the most staunch privacy advocates. They're like, wait a minute, if you have a server that's in your infrastructure and has access to my email, then it's really not end to end, it's not encrypted. So we need to figure out a good solution to that before we can explore that. But some other things that I alluded to earlier involve expanding. That's such an overloaded word. Expanding the context that the model has access to. I don't mean like context window, I mean like, okay, so giving it access to your calendar, to your, to dos, your RSS feed. What if we added a notes application to Thunderbird and then. And effectively turning. Making it possible so that Thunderbird could be used as an LLM assisted personal knowledge management and communication tool that whatever future that looks like, that's more exciting to me personally. I'm one of those people who like, I have notes from the last handful of like last couple of decades that I still keep around and I would love an LLM to help me sift through that. It would be even more interesting that as I'm making a note, it could suggest related documents and ideas that I've had in the past or just, I mean, for a lot of users, just helping them stay organized. Because again, there's so much for one tiny human brain to keep track of and there's just so much information. So I think that for me as not as an ML researcher or an AI expert, I'm just an application developer, I want to work on that. I want to build that and make it possible for people to have more control over their information, help them retain their privacy, but, you know, make those creative connections that only they as a human can do. But an LLM local or, you know, confidential, remote compute assisted, reminding you of, like, oh, you wrote this. Here are some things that you've written about that are related to that, or here's some conversations you had in email or in chat. And then for you, the user as just a regular squishy brained human, you're like, oh, I just had this weird random flash of insight based on this constellation of information that I generated over years. I think I like that future of AI. And also as an application developer, I think that I really want LLMs to be more deterministic. It's so weird to call an API with the same data and get very different results. And we can get into this or not, but I definitely feel like chat is the wrong interface for a lot of tasks.

B (50:06)

Yes, thank you.

D (50:07)

Okay, cool. Just want to make sure I'm in good company. So I've got, I don't know again, in my little Mad Science notebook. I've got ideas around like, okay, how do you swap in deterministic functions? How do you coordinate the efforts? And I think something beyond. I mean, maybe I'm describing a more strict version of mcp, but the fact that your input and your output currently is plain language is. It's a double edged sword because the only way to determine if you've got a bad result is for you as a human to evaluate it, unless you spin up another language model to verify the first result. But as a programmer, that feels a lot to me, like, oh, I just wrote a function and the only way to know that if my function call was correct is to write another function to check it. And it just feels like. It just feels wrong. So I really, yeah, I want discrete inputs and outputs. I want language models that are small and dedicated to specific tasks. And then I want reusable, shareable ways of wiring them together. I want to create essentially like workflows of information processing within Thunderbird. So that's me. I'm the personal knowledge management cheerleader at Thunderbird. That's my new title.

B (51:41)

That's awesome. Well, that future is one that I could get on board with for sure after struggling with a lot of the things that you mentioned as well and also hoping for many of those things. So, yeah, thank you so much for sharing this journey and this experimentation that you've been on with Thunderbird. And please keep up the Good work. Give our thanks to the to the team for inspiring us with a lot of amazing work. And thanks for sharing your insights here with us. Appreciate you taking time.

D (52:14)

Yeah, thank you so much for having me. I'm really, really glad that we could make this happen.

B (52:20)

Us too. We'll see you soon.

D (52:21)

All right, thank you.

A (52:30)

Alright, that's our show for this week. If you haven't checked out our website, head to PracticalAI FM and be sure to connect with us on LinkedIn X or BlueSky. You'll see us posting insights related to the latest AI developments and we would love for you to join the conversation. Thanks to our partner, Prediction Guard for providing operational support for the show. Check them out@prictionsguard.com also thanks to Breakmaster Cylinder for the beginning and to you for listening. That's all for now, but you'll hear from us again next week.