← Software Engineering Daily
Software Engineering Daily
Palantir with Akshay Krishnaswamy and Christopher Jeganathan
00:51:03
Palantir Technologies is a data analytics and software company specializing in building platforms for integrating, analyzing, and visualizing large datasets. The company’s tools are designed to help analysts and decision-makers collaborate on data-driv...
Loading summary
Transcript63 lines
- [00:00]
Akshay Krishnaswamy
Palantir Technologies is a data analytics and software company specializing in building platforms for integrating, analyzing and visualizing large data sets. The company's tools are designed to help analysts and decision makers collaborate on data driven solutions to complex problems and they have worked extensively across the intelligence, defense and commercial sectors. Akshay Krishnaswamy is the chief architect at Palantir and Chris Jagannathan is a group lead at Palantir. They join the podcast to talk about the evolution of Palantir, its technology, the AIP platform, and more. This episode is hosted by Sean Falconer. Check the show notes for more information on Sean's work and where to find him.
- [00:52]
Sean Falconer
Akshay Jag, welcome to the show.
- [00:54]
Chris Jagannathan
Thanks John. Thanks for having us.
- [00:56]
Sean Falconer
Yeah, awesome. So Akshay, you've been at Palantir a long time, I think around 12 years. So I wanted to get a little bit of background on the company, like its area of focus, how it's changed during your time. For anybody who maybe is less familiar.
- [01:10]
Chris Jagannathan
With Palantir, yeah, I think it's always a tall order to try to summarize things, but I guess being here for 12 years makes me a bit of a Palantir boomer, I think in at least relative terms. Maybe just to give the very abridged version. The company was founded in the aftermath of September 11, very much focused on a set of counterterroris emissions across US and allied governments. Kind of the principal problem at first was how can you solve the fragmented data problem to be able to make better decisions in the context of certain national security missions. And it turned out like there was a lot of technology that had to be built around data integration and being able to enable frontline analysts and soldiers to be able to make better decisions. And kind of the core technology there around being able to integrate data and then link it to operations ended up being kind of this like common theme which took us throughout other parts of the government sector. So now we're kind of all across, not just the defense side, but also, you know, Veterans affairs, cdc, working with the NHS in England, which Jake probably knows more about than I do, but also that kind of kernel of technology and the ontology kind of concept and system that we built from there took different renditions as we went to the commercial market, first in the financial sector, other areas where it was a lot of regulation, a lot of fragmented data, and then into, you know, domains in like manufacturing with Airbus, bp, et cetera, and kind of all across the commercial sector since then. So it's been kind of this process of like being proximate to problems, seeing the complexity in them from a data and an operations perspective and then building these platforms kind of, you could argue backwards from that.
- [02:36]
Sean Falconer
Yeah. And the company itself has been around, as you mentioned, like founded after 9, 11, so I think it was like 2003 is around the like official founding date. So now over 20 years. This probably started as I would suspect, like an actual desktop application and then moved over time to the cloud. So there's been a lot of transformations that have happened from the technology standpoint in order to essentially modernize. Right. As well as sort of the business standpoint where you started to move more commercial, outside of government for sure.
- [03:02]
Chris Jagannathan
Maybe I can give a little bit of that Boomer context and JAG can give a little bit of the modern version of how we've evolved to the current stack of technologies. But yeah, you're right, it started off as a set of Java services and a Java Swing client. And I still remember one of the first things we had to do early on was like how do you debug the runtime errors on people's particular machines to get them to actually use the thick client correctly? And there's a whole different set of issues when you have people on rugged laptops running all the services local in a disconnected environment. And it's like, what's the boot sequence? How do we get the logs off this thing? If it's a security gap, how do you do that? And it's like a lot of the infrastructure we built kind of to enable more flexible deployment options. And also like the current experience of using the platform largely through a browser based set of interfaces came from a lot of the pain of saying, okay, how do you build these things early on using the technologies at the time? And there's a whole journey there. And maybe Jack can talk about this too of like transitioning from those technologies where we built a lot of sophistication for those tool sets into more modern tool sets over the past several years, while not like sort of leaving people in a lurch who had all those requirements kind of build up over the years. So maybe Jeg, you can add some color to the current stack too.
- [04:07]
Akshay Krishnaswamy
Yeah, I find it interesting because I joined about five years ago and I remember even in my onboarding we were still using the Java Swing client to use the product. And that Swing client has only very recently been completely removed from our product and we've moved to a more standard web stack. But it's been quite interesting from this idea of having all these services running on a user's end laptop. To us now thinking one of our customers somewhere probably has a 2013 ThinkPad that has no memory to it. How do we get these products to run on any device that our users are using? And then I often joke that we've gone full circle, that we were talking about how we were moving to the cloud and now you can see our software running on trucks, on the edge, and in any environment that we can get them to. But ultimately, yeah, we've definitely moved a long way. I never got to experience the golden swing days myself. Like, everything we do right now is web based, but still we are now coming back to that story of can we run in any environment, whether it's on the edge? I think that's been like consistently in the DNA of what we've produced.
- [05:15]
Sean Falconer
Yeah, I wrote more swing code than I'm proud of. I'm glad that those days are now behind us for the most part. But I would think that, you know, what are the update cycles like when you're deploying some of these things to, you know, government and military and these scenarios? Like, I think when I think about government, a lot of times I think about like, they're maybe cautious, they're not necessarily going to do the latest update because of if something's working. Like, there's always risk inherent with updating. So are you always in a situation where you're dealing with an older, older hardware, older operating system, these types of constraints?
- [05:46]
Chris Jagannathan
Yeah, there's. And maybe I'll again turn it over to JEG to give the more detailed version here. But a lot of, I think the complexity there that you talk about, Sean, is why we built our Apollo platform. So kind of you think about the kind of user facing platforms historically. First it was Gotham, which has evolved now into a set of defense offerings that we kind of broadly call like the Defense suite. So it's like it's the core Gotham workspace, which is now web based. It's Meta Constellation, which is kind of more satellite imagery oriented, and a few others as well. Then there's Foundry, which is kind of the broad based platform which now even Defense offerings sit on top of. Then there's aip. All of these are like microservices architectures which are now backed by the Apollo platform, the continuous delivery platform. And you know, I think continuous delivery is one thing, right? Like, how do you ship code incrementally? How do you deal with somewhat different constraints and environments? But to your point, if you have regulatory reasons, why you can only upgrade in certain cadences if there's like a complete kind of tool chain of additional requirements that have to go and validate the binaries you're shipping. Beyond that, how do you, you know, still get the efficiencies of scale without having to build one off forks of your delivery platform for every single customer? And then how do you deal with the fact that like you might have to do all of this continuous delivery in a disconnected hub that exists in a classified network or somewhere else? And so what we try to do is get as close as we can to what we can do for commercial customers typically, which is like just kind of transparent SaaS based commercial delivery. Right? Or continuous delivery of software updates for every part of the service mesh. Apollo allows us to then say, well, what are the risk tolerances you want to impose and how do you want to make that happen? And let's say say a more sensitive government environment, even a manufacturing environment in commercial where you might say, okay, there's blue, green deployment channels of course, and there's like ways in which we can have soak times for new versions of services, then there's additional checks that the agency or whoever else has to do on top of those new service versions. Then they can get automatically promoted to a new branch or a new version. And essentially this was like all us doing this in the field as forward deployed engineers, right? We had to build, as our CTO calls it, like robots in the software instead to do this stuff. And so actually what's cool is Apollo as a platform that kind of does all of these different requirements, gating, promotion, rollback, security, patching, et cetera, is now being used by some of our commercial partners like Cisco and others to deploy their own software in these complex kind of heterogeneous environments. And maybe JEG, as one of our development leads, you can give a sense of what the life is like now with Apollo working at scale.
- [08:03]
Akshay Krishnaswamy
Yeah, I have to admit, Apollo makes my life ridiculously easy. I don't really have to think about what I'm doing as a developer. For example, on our front ends where I normally spend most of my time, we make a cut of our code every Monday and Thursday and then we have various soak times in different environments. And with Apollo, I can very easily go speak to one of our customers. They'll say, oh, when am I going to get this feature? And I can predictably say, oh, have a look on Tuesday. I'm pretty sure it'll be there. We can fast forward delivery whenever there's a requirement from the customer, but we can also, like, freeze their stacks if they need it. Like, all of our customers have different requirements. And as we were saying, like, that tolerance to risk, where, I don't know, if you're just developing on the platform and you just want the latest thing, we can get you, like new versions of code within two days of it being written. But in the environments in which you have change, freezes and all of these, like, SDLC processes, Apollo really lets us capture exactly what that customer needs. So the developers on our side are kind of just coding, getting their stuff shipped and then worrying about, you know, the code getting to the customer, very little, just telling them their timelines. It's a very easy life.
- [09:16]
Sean Falconer
And then in terms of given these constraints of some of these environments that you have to deploy to in, like, the world of, like government or military, like, have working with those types of agencies under those constraints, like, helped inform parts of essentially the commercial product and how you think about, like, building products there or even had been able to solve things for, you know, a commercial company that you wouldn't have been able to solve if you hadn't had that experience.
- [09:41]
Chris Jagannathan
Yeah, absolutely. There are a couple of things stand out. So I think early on in the government space, it was sort of there were two pillars to the technology. Early on there was the data integration piece, which was like, how do you deal with all these different forms of data coming in, structured data, unstructured data, even at the time, like, it was documents and imagery and things like that. How do you reconcile them together inside of a common, essentially world model, like an ontology that you're building? So you have a kind of a common substrate that these technical teams are working on to enhance and then is kind of immediately accessible to different types of business users or operational teams. And that then necessitates the second thing, which is having a security model that can be super granular, right? So it's like, it's not just the fact that I have these objects or these links, but it's, you know, which version of the object. Like maybe JAG added that this person's eye color is blue and eyes that it's brown. And you're our manager, Sean, so you can see, like both things where we can't see each other's and it's the same object ultimately. And all those things, I think through architectural pain, I would say we got to a point where we understood, like, how do you build some of those core frameworks in place that can be generically applicable? So for Modeling, like mortgage fraud or oil and gas work, or the A350 assembly, it's like you're kind of continuously learning. Okay. Like, there are some of those lessons from the integration space and from the security space that make it possible now for us to function at a place like Airbus, which has operations split across Spain, Hamburg, Toulouse, and has all probably the most regulated manufacturing environment in some ways in the world. And then I think from those environments too, I'd say vice versa is true. So, you know, when we say, okay, at a place like Airbus or at a place like the Cleveland Clinic, you see new requirements around how the applications have to be able to capture user context and then feedback to the ontology and allow technical users to reconcile those changes that then gets shipped back. Like, because it's all part of the same platform, the government users benefit from that work. Right. So I think it started off in the direction you're talking about. What's cool to see now is it's kind of like the kind of flow lines come in from both sides of the business. And maybe Jack can add to that too.
- [11:34]
Akshay Krishnaswamy
Yeah, I definitely saw this and I was talking to a customer two weeks ago, in fact, and maybe I don't have the right picture, but this customer was there like, do you know what the thing that you clearly learned from government is that you built everything with permission as a primitive. And I was like, oh, well, I've not really thought about it, but it makes a lot of sense because it's just how we think day to day, that, like, permissions were the first thing that entered our platform because we were looking at government users. And then we added the feature set on top where a lot of other companies are doing, like, features and then working out how to apply security around them. And even the way that we talk about how we build the primitives and the concepts within the platform, like the language that's used internally is always around the security model and our provenance systems. And I think the fact that our whole system is grounded in this concept of provenance for every piece of data within the system, wherever you're using it, whether you're using it in AI or in data integration or in our ontology, the first thing we want users to know is exactly where that data has gone, where it's being used from, and what the lineage is. We absolutely love our graphs here. It makes a huge difference to, one, the way we make our product, and two, the way our users can consume it and apply their own software development lifecycle processes. All of the Things that they care about once they're starting to scale their systems up.
- [12:57]
Sean Falconer
Yeah, I mean, I think that it sounds like you took a very like first principles approach to making this sort of secure by default, because you had to, essentially based on who you're selling into. But it's very difficult, I think, for companies that are maybe coming from the commercial world and then hadn't really thought about that, and then they try to layer these things on. And I think that's kind of where they end up getting into trouble from a security perspective because it's just very difficult to do that kind of after the fact and do it correctly unless you really put the investment in.
- [13:24]
Chris Jagannathan
That's right. And I think even being candid about our own journey, it's like we built an initial security model in the first versions of Gotham that scaled so far. Right. It sort of had this idea of role based controls, more granular controls within the expressiveness of that current ontology, which is mostly objects, links, properties and things I would say are more semantic elements, kind of in the traditional sense. You expand that now to the kinetic behaviors you're modeling too. So it's actions that are backed by arbitrary logic. Like Jeg was saying, it's like we need to be able to model holistically if an AI technique or function is interacting with any of that stuff. And so it required us to actually upgrade and build new architectures under the security model over time. And I think we all remember kind of different points in the scar tissue of like, oh, that was when this particular iteration didn't properly scale. Right. Or like it was scaling up to the point. But now when you really turn the spigot on when it comes to users doing these sorts of operations with like much more granular statistics or something, that suddenly blows up the count of this type of security tabulation. Like, we have to think about how to address that again. So I think because we have context, I think frankly, that have always kept us honest about like, you can't release a new feature here unless it abides with a security model that forces us to keep working on the underlying model and the provenance service and all these things, like JAG said, because we literally couldn't ship these things otherwise.
- [14:39]
Sean Falconer
So you mentioned ontology a couple of times. And that's really how sort of Palantir originally got on my radar was I was working in the world of ontologies during my PhD in postdoc work. But outside of Palantir, I think like no one outside the academic world I think relies on ontologies at the same level as maybe Palantir does? Maybe can you give a little bit of background on like how you're using ontologies and like what's that help inform within the Palantir platform?
- [15:04]
Chris Jagannathan
Yeah, maybe I can give again kind of the history lesson here and then Jag, I'd love, I'd love your take too. But it's like I think only a company that had no marketing team for 20 years could have probably, you know, built the entire technical story around a term like ontology. I think there are people like yourself who appreciate it, but it's like it's always have a philosopher as our CEO and founder, but it's like I think it came from this recognition that if we're going to integrate data and we're going to make it accessible to operational teams, like what is the commonality between essentially the technical work or the analytical work and the work that's actually mission driven? And I think the idea was like if you represent the world, call that an ontology, it started off by saying okay, they're the people, the places, the things, the objects, the links, the world that I think was honestly fun first there to help people with investigations and what you might think of as read heavy operations with some amount of write associated with it. I think what was interesting and I think what we probably couldn't have predicted but the additional mission context and really this was like the transition from intelligence workflows to more forward deployed war fighting workflows, much more operational, much more kinetic and then the work in the commercial sector was much more industrial and manufacturing oriented, forced us to evolve what that ontology system did. And actually the foundry platform took the original Gotham ontology idea and actually reimplemented it. And now that is the common ontology system for all the platforms. But it expanded on the, you know, expressiveness of those core objects, links and kind of traditional kind of read oriented elements with these ideas of co modeling behavior as well. So I think the way that we think about it is like it's a decision centric system now and that's kind of buzzwordy. But what we mean by that is like it's modeling the data, the logic and the action together. So it's like what is the information at my disposal if I'm looking at a supply chain reallocation strategy? It's like there are semantic elements there that make up the world of the packages, the customers who might be shorting me on the supplier side, the customers, et cetera. Like on a graph. We Love our graphs, as JAG said, or something else. But then it's like, what can I now do about this? And it's like, there are sets of behaviors here that maybe come from business logic in my ERP system, or maybe I've defined the rules for inside the foundry system that I want to be able to then evaluate, and maybe those are contextual. So there's a graph that's like the data graph, and now there's like a state machine of what I, I can do. And so this is, I think, part of the kind of like overwrought description you'll sometimes hear from Palantir of like, what is the ontology? It's like, well, there's like a data element, but also there's like this workflow and state machine element to what you're defining in the behaviors. And then those two things can be represented to AI systems as well if you want them to help you orchestrate or do any part of that process. And so it ends up being, I think this combination, I think the way I would describe it to most folks is like, it's kind of like the object layer that I think people talked about in the era of like Xerox PARC and small talk, or if you think about sun and Next in the 90s, it was like, what does an enterprise object system look like in the grandest sense of the word? I think that's what the ontology feels like to me.
- [17:49]
Sean Falconer
Now, I'm assuming you have probably some baseline ontology, and then on a per customer basis, are they building essentially instantiation? It's like an abstract class and then someone's building their own defined class for their specific use case and their understanding of the world and their business requirements.
- [18:07]
Chris Jagannathan
Yeah. And again, just being candidates, like, we've kind of gone through many iterations of what those archetypes or those abstract classes could look like. And I think we're always kind of skeptical of them, to be honest. It's like, because you show up with like a starter pack. It's like, here's like the ontology. And of course, if you work in like, I think you come from a bioinformatics background, like there are disease ontologies or things that you just need to have if you're working in certain spaces that are like table stakes. But I think, like, we always think about those as a starting point because what you're really layering in because of the combination of data and behavior is like the specific kind of like workflow dynamics of your enterprise, which are always hard to think about in the generic. And so I think like we always say like, here's like a starter pack. You're doing quality work in automotive, you're doing like, you know, flight work in aerospace. But no, it's just a starting point for you then to be able to evolve and make it your own through a use case or two. And so I think we're probably, we come in and say, cool, let's like start with this default starter pack. But the first ontology will emerge once we've implemented a first use case. This is how we think about it typically. And jag. I'm not sure if you have other thoughts there.
- [19:05]
Akshay Krishnaswamy
No, I think you kind of hit the nail on the head that like if we often just come in with what we think a business will look like, even if it's been successful with another business. I've always seen that ontology be mutated in various ways because the businesses we arrive at and the customers, they already have a way of thinking, a way of working and their own articulation of their world. So for one automotive industry, they might have a different ontology explicitly to another. We can try our best, but we're not prescribing exactly how to run your own organization.
- [19:40]
Sean Falconer
Got it. Introducing height, the only autonomous project management tool. Backlog grooming, bug triage, keeping documentation up to date. Those aren't why you got into product building, right? Well, height handles all that grunt work for you. Using a first of its kind AI approach, height proactively takes care of time consuming workflows without you lifting a finger. Height recognizes when you've agreed to trim, scope and handles, mapping the necessary edits back to your product brief. When new tickets are added to your backlog, height combs through them, adding feature tags, time estimates and more. And it's not just you. Everyone on your team manages projects, tracking updates, scoping work, balancing priorities. But whether or not your product succeeds shouldn't depend on project management. With height, autonomous workflows handle that mundane upkeep so your team can focus on building great products. If you're ready to stop managing projects, it's time for height. Join the new era of product building where projects manage themselves. Visit Height app Sedaily to get started. So, you know, we've been talking about some of the history at Valentine. Some of the transformations kind of have gone through from, you know, regionally very defense and government focused to being commercial. The dairy are some of the platforms that you built, I think feels like to me that you're kind of going through Another transformation now to becoming like an actual developer platform with things like aip, which is your AI platform. I mean, how would you describe aip just to start with?
- [21:03]
Chris Jagannathan
Yeah, maybe I can take that one. Then Jay, you can talk about the developer platform piece. So AIP was very much the recognition of like, if we think about the existing set of services we had built in the foundry platform, around data integration, around the core ontology, around the application building layer. It's like, hey, a lot of this is like it revolves around this idea of modeling the enterprise, this ontology system which is modeling, like we said, the objects and the data, but also the behaviors. And I think AIP is the idea that you can connect all sorts of generative AI models. That was the conviction. You can take the power of these emerging models. And we had seen some antecedents in the government space that were not quite what we saw in the commercial sector, but kind of gave us a hint of where things were going. And it's like, well, if these models have these kind of reasoning capabilities, they're going to need to interact with the data, the logic and the action in the enterprise to be useful. It might be helping us just structure data, it might be helping us essentially in a copilot function, look for different types of pathways we might evaluate in the workflow. And then maybe at the limit it's actually an agentic workflow where it's working across the entire state machine or finding new avenues within the security boundaries and limitations we would affix to users.
- [22:04]
Sean Falconer
Right.
- [22:05]
Chris Jagannathan
And I think it was kind of this moment of somewhat semi serendipity of saying, well, a lot of what we had built because of this very heterogeneous collaborative model of different types of users interacting with this ontology across enterprises we've been working with like if you just kind of model AI as another user at the table, AIP was kind of the entire set of infrastructure needed to enable that. So it was like the language model service we built to be able to interact with all sorts of different models securely. So it's like we love our partners at azure with the OpenAI services and anthropic through AWS and the Google Gemini models. We have our own hosted model hubs for the open source models. You can bring your own, register them. And the idea is like, can you provide the platform? I think this gets the dev platform side for allowing people to build AI enabled applications that are fundamentally going through that ontology system not just as a semantic source of grounding, but also as a security and governance system to JEG's prior points around like provenance and things like that. And I think that did kind of segue as we caught wind with that to the dev platform side of things too.
- [23:00]
Akshay Krishnaswamy
Yeah, I guess I can jump on the dev platform side of this, which is really describing the full journey that we've gone through where we started building this product at customers, with governments, where we were literally just trying to make tooling for ourselves. We wanted to enable our most technical employees to be able to build things. Until we found that this definition of an ontology actually made it very easy for our operational users to interact with the system, to understand the semantics and the kinetics of their business and build things using our no code tooling. And we kind of came full circle where all of those things we're describing like ontology get you very close to the HCI principles that we've been thinking about for a long time. And object oriented programming, where you're describing the model and then it's the thing you actually want to interact with. When you're building software on top, the idea of collecting the exact objects that you want to interact with, which are a subsection of your business, and then getting the actions that you want to perform on those and calling them as if they were methods, is actually very attractive, especially as your business scales, because you're not thinking about that same modeling problem over and over again when you're doing software engineering. We literally have like a shopping cart experience of like, I'd like these objects today and these actions and maybe these AI enabled functions. And this is exactly how I'm going to power this particular interface, decision, product, maybe application. So it kind of. I like the term semi serendipity because it carefully highlights the point that if we called it serendipity, a load of engineers here would be annoyed about the amount of effort they put in. But the semi serendipity of us actually getting to a developer platform seemed very obvious once we got to the point where we were like, yeah, now we want to code against it.
- [24:47]
Sean Falconer
How long has Palantir been investing in large language models?
- [24:52]
Chris Jagannathan
So I think we all kind of collectively caught wind of things, I think, after the release of ChatGPT. So a former Palantir director of engineering, Bob McGrew, actually was the head of research at OpenAI. And Bob was doing amazing, awesome work there until quite recently. And he's moving on to his next thing, I think publicly he said. So I think we kind of like caught on around the same time, I think really in earnest that most people did with the launch of ChatGPT. I'd say there were a couple of things that were antecedent to that. One was, I think we saw some. We were kind of following some of the literature and some of, I'd say, the precursors in the transformer space because of our work in DoD and other fields and just like staying attuned to things, we also had some sense of what was going on at a high level at OpenAI in other places, just because they were all publishing most of their work, right. It was even open source up until through GPT2. And so I'd say, like, we got serious about it, probably with the sense of, like, we have the conviction to try to build something here. Shortly after, I'd say, like that fall, that winter, when the kind of chatgpt boom happened, right? And we said, well, now that there's really a there there, it kind of got to a point of congealing internally where it said, okay, there's all these things we're also building in the foundry system that will, I think, coalesce and come together to enable something really interesting that would merit something like aip.
- [26:02]
Sean Falconer
Did that feel like a natural fit based on some of the things that you had already previously been doing, you know, combining maybe traditional machine learning and AI approaches with everything else that you were doing in terms of these various platforms?
- [26:16]
Akshay Krishnaswamy
I can definitely talk to this just because I had the opportunity to work on some of our earliest AIP pieces, our initial interactions. And I think it was very much we were seeing this technology come through, and it just aligned so well with the ontology concepts that those were the things we wanted to try. We wanted to see, like, does retrieval, augmented generation work with objects when we put them together? And I think many people and like, engineers and like four deployed engineers alike, entered this sort of research mode, where it was very much like, out of sheer curiosity. Do these things connect well together? What happens when we start defining systems where we, like Akshay said, create AIs the next player in the multiplayer environment? And then as soon as we saw that, actually, when you're using these products initially, we're trying to get people to a point of decision making where you don't normally get these with a large language model, where you're like, where did my citation come from? What is the actual meaning behind this decision? Can I actually go through the working out? We found it to be a lot more clear when we could say, like, cool, this is the corpus. This is the truth of the World. If we give you this truth, you have to refer back to the truth and tell us exactly how you came to a decision or what comes next. So by giving our users and the AI the exact same image of the world and saying, cool, this is how I came to my decision. And these are the constraints we found very consistently in multiple workflows that this came together very well. And then we just started iterating very explicitly on how do we build trust in a system that's using something that is non deterministic by nature, but enables people to do their workflows so much faster.
- [28:02]
Sean Falconer
What's it like building on aip? Maybe Jack, you can walk me through, like, what is the equivalent of like building the hello World application on this platform? Like, what's it kind of take to get started as someone like fresh to it?
- [28:14]
Akshay Krishnaswamy
Yeah, it's interesting because I recently thought I'd make a notes app. I was there like, what is the hello World here?
- [28:21]
Sean Falconer
Yeah, like a tic tac toe notes app. You know, the classic get starter projects.
- [28:25]
Akshay Krishnaswamy
Exactly. I was trying to work out like, what's the hello world of AI? And now we have like the idea of retrieval augmented generation. Can I make a notes app that when I type in a question will tell me an answer, but go like, hey, look at this note where I found your answer. So what am I having for dinner? Well, your groceries has chicken on it. So I guess you're having chicken. And getting started then in this narrative is like, cool. You make the notes app just like you did before. Like we're not taking that away from the engineering experience, it's how you augment it. And you use our internal tools in AIP to define the logic as you walk through, like, what is a note? How do I bring a note into the application? What is the question I want to ask? How do I embed into a semantic database? But in our platform it's very much like Qlik interactions where I don't know how many times you've tried spinning up your vector database at home just because you wanted to try semantic embeddings and then connecting to various APIs. But internally these things that are the basic primitives of AI are just baked in. So then when you're writing your code, it's cool. I'm going to call this endpoint. Tell it that I'm working with note objects which are already defined in my object model and I want it to tell me exactly based on the content. Can you find a note that matches and give me information? So we Very much expose the APIs that are the primitives within the AI platform and within the gen AI space that people are using day to day, whether it's vision or whether it's semantic searches, whether it's rag, and then allow you to just define them as an SDK that gets generated for you. So instead of having to think through like, oh, what is the infrastructure I'm going to spin up today because there's a new white paper on some new technology, or there's a new language model with a different multimodal interface. It's very much cool. I have an SDK I can call it in Typescript. I'm going to just use it in my Notes app and not think about it as a difficult problem. In the same way that making a to do app isn't a difficult problem.
- [30:30]
Sean Falconer
Does the platform have an opinion about how some of these different components, these base units of AI work So that as someone who just wants to essentially produce this Notes app, I don't have to think about what is my text chunking strategy. Those things are just an abstraction. I have an API endpoint that I'm calling.
- [30:51]
Akshay Krishnaswamy
Yeah, exactly. So we have very easy mode versions of these things. I describe it as like, for every interaction you want to do normally, then there's usually like a one click, Hey, I want semantic embedding or Hey, I want to do text extraction. And then there's usually a dropdown somewhere, which is cool. Now every time I talk to a set of customers, someone always asks, can I adjust the seed? I personally have never changed the seed in my life, but I know you can change the seed is these kinds of interactions where like, the more competent you are and the more of the work you want to expose to yourself, especially like the interfacing, you can do that all the way down to like, if you want to hand code the interactions that you have with a new language model and then register that as a contract within our products and be the other side of the wall, you can bring that as an interface. So the way I've kind of experienced it myself, because I got myself one of our free accounts just to see whether I'd enjoy it or not, is you can very much build exactly what's worth building very quickly and go like, cool, this is the USP of the thing I want to make. And then for every individual component that you are adding to your system, you can decide, is this worth as a microservice pulling out and writing it myself, or should I just use the Palantir default for every single step. So if I just want to use like the default embedding model and not think about it, because that isn't the most valuable thing for me to do, then I can do that. And then the next day I can go and I can spend a day researching what the latest in embeddings is.
- [32:31]
Sean Falconer
How much of it is platform driven. Am I building an application? Am I in control of what I'm using on the front end? Like, can I go and use React or Angular or whatever framework I want to use there? And then I'm slowly using AIP for sort of the API layer and the back end. Like, can you kind of explain a little bit about what are all the different components and how much of it is the platform versus things that I have to build myself?
- [32:57]
Akshay Krishnaswamy
Yeah, I mean, you're fully in control. Basically we provide a vite template that will build your, like, classic React template or Next js. But at the end of the day, what we're really exposing are restful APIs, like we're doing the most raw integration that we can do with the platform, so that if you turn up and you wanted to write all of your code in Rust, that's your prerogative. I wouldn't recommend it for a front end application, but it might be something that you want to do. And at the end of the day, we provide SDKs on standard languages like Python, Typescript, Java. But we are fully giving you the opinionation onto how you want to build your product, whether you have an existing application that you just want to add AIP to, whether you want to use Angular, React, Vue. Ultimately, we will provide opinions because as Palantir, we have opinions as to how you could get started, how we can make your life a lot easier, what we do in any environment. But we've had customers just building things that we hadn't thought of or building the first version of something because they were insistent that the first time they were going to use AIP was going to be in an Expo app. And I had someone going like, hey, can you set up an Expo app? And I was like, I've never made one, but let's work through it together.
- [34:17]
Chris Jagannathan
Part of what I think Jag said earlier as well, around the things that were coming together around the time of aip, I think it reflected a lot of learnings and requirements from enterprises as well, who said, if I'm going to rely on this ontology layer or this ontology system, really it's like you need to provide me the leverage around Being able to use different data computing compute engines in the pipelining layer. Like I want the provenance service, I want the security, I want all these things, but I want to not only be able to use Spark and Flink, but also be able to use like lightweight transforms like Polars and I want maybe Clickhouse or an open source engine or something as well and plug that in in a way that actually is seamlessly integrated with all those things I like about the integrated system. Same thing like you said. I think earlier you said, Sean, like, were there other types of models that kind of forced us in a way to have some of the infrastructure ready for the AI, the gen AI models. And that was very much true. It was not just traditional machine learning models that people were building and dockerizing and they wanted to bring into the platform or hit externally, but it was also optimization models, right? The things that are like solvers that are being used to run GAS networks or airplane networks or things like that. So how do we have you able to have an ontology action that is backed by one of those models or any type of model kind of generically with some sort of contract? You're defining all of that complexity, right? Being able to kind of mix and match and evolve these things over time. I think now comes the benefit of the developer platform where you can come in and say there are lots of sensible def defaults, like Jag said, but if I want to slot in and out all these different types of data and compute tech within a common perimeter and a common interface through the ontology SDK, I think the experience has been it provides a lot more leverage than like Jack said, having to do all that stuff yourself.
- [35:49]
Sean Falconer
How much does plug and play? Like, can I run my own, like you know, private open source model or you know, plug in my own vector database if I'm doing something with rag?
- [35:58]
Chris Jagannathan
Yeah, absolutely. And maybe Jack can give some of the patterns here. But it's like, I think one of the things that like this has gone, I think very deep again because of the requirements from 55 plus sectors now at scale, it's like people have their existing data platforms even so it's like can I virtualize data from the existing Snowflake databricks bigquery instance I have, Can I bring in the metadata from my informatica system or my elation system or something else? My semantic definitions maybe sit inside of some OWL file somewhere? So can I bring those in and map Those into the APIs? Jack mentioned the model building. Yeah, it's like I Want to connect and pull open source models from hugging face. I want to bring my own models from my existing data IQ cluster or my, you know, my data robot cluster, whatever it is, or my just my EC2 environment. All those things are kind of like pads that we've had to make smoother and smoother even though they were technically possible, because those are the things that people, all every enterprise architecture team is asking about. And I think what's cool is like when the developer platform kind of turned on, by no means perfect, still working on it. It's like people got a lot of that leverage from the years of having to build all those on ramps for different types of integration patterns.
- [36:58]
Akshay Krishnaswamy
Yeah, I think it is very interesting to describe the journey of when we were asking people to do data integration originally we were very explicitly, you must use this language and you must use these technologies. For a long time we were like, you're writing Python and you're writing Spark, and that's the journey of data integration. And then slowly, over time, we've exposed more and more of the infrastructure layers so that people could have that level of customization. So you're not just picking your language or picking the environment in which your computers run. But we've gone as far as going like, cool, you can bring your own containers. And then our job now becomes like, how do we show you very natural interface points so when you bring a container in, then it's treated correctly. In our provenance service, you can see it being used in your ontology. You can define an action that runs through some logic that maybe Your business wrote 40 years ago and refuses to change because they don't have any engineers who can, who can write cobol, for example.
- [37:56]
Sean Falconer
Right.
- [37:56]
Akshay Krishnaswamy
These are like real problems that we've had to solve and face.
- [38:00]
Sean Falconer
Now you have your LLM write the cobol, right?
- [38:02]
Akshay Krishnaswamy
Yeah, it's true.
- [38:03]
Chris Jagannathan
I was just talking to somebody who was like, if, you know, like all you people now who have these LLMs, I had to do this by hand two years ago.
- [38:10]
Sean Falconer
Get off my lawn. Right.
- [38:13]
Akshay Krishnaswamy
You do have to admit the engineering problems have become a lot easier now. I feel like I write less code myself, but ultimately, yeah, we can bring your containers, we can bring your front end applications. Our egress to other services is completely designed around just making it native to the things that are in your ontology. Because in reality, if everything was inside aip, then your business isn't actually running right. It's like the connective tissue between our products and exactly the things that are your system of record things that actually mean that something changes in your organization means that we've had to force ourselves to integrate with every product under the sun, whether we've liked it or not.
- [38:56]
Sean Falconer
How does having the ontology at the core of a lot of this help with things that are challenges with LLMs today, like hallucinations, you know, lack of domain knowledge, some of the security and governance over the data.
- [39:09]
Chris Jagannathan
Jag mentioned a couple of things that I think are kind of specific elements there and how I think the ontology services are grounding system for the LLMs. I think the core thing though, the core kind of central assumption is it's the same image of the world. Like Jeg said, it's the same rendition of the world between humans and AI. So when you have to go and look up objects and run object queries using the object set service, or you have to invoke an action through the ontology system as a human user, there are kind of all these junction points, right of like, here's how the provenance service tracks that, here's how the action gets run here, how the results get staged or in a scenario or get executed and then written back to an SAP system or somewhere else. And all of those things have like kind of defined behaviors, protocols, frameworks around them. And so like when we introduce the AI capability here and you're defining an AI function that is, you know, looking through a support ticket, figuring out which customer orders might relate to it. Like you're literally saying here are the customer order like object tools that can interact with, that are whitelisted for this particular function. Here's the actions it might consider. These are all actions defined in the ontology. And I can literally like run these dry runs even as and this is kind of the integrated development environment in the AIP logic application. But you can also do it programmatically. It just kind of shows you here is like how it's interacting, interacting with all of those different object based tools or I'm sorry, ontology based tools, objects, actions, et cetera, in the same way a human, like I could literally go and do those same exact things. And I think like being able to have every single run, every permutation of the interaction, whether it's just feeding information or it's doing something agentic, always be in that same system, that same set of guardrails that the human users are all working within actually serves as the most natural on ramp for introducing these capabilities in different customer settings. So I think about like some of the just very quick customer examples like, like Eaton, a big manufacturer does, like material reallocation and supply shortage disruption, handling all the time with human users. And they built this ontology for doing that. And they could say, well, there's actually this part of the workflow where the LLM could help us do better retrieval or turn our alert inbox into a solution inbox because it's proposing things we might do. And that was just something we were already doing. It's just coming into the workflow as like, one Japanese customer called it, like the intern on the team. The intern knows nothing. But if we give it some context, it can maybe be kind of useful. You wouldn't trust an army of interns to do everything, but they can do some things and then like a nurse scheduling is another example at the Cleveland Clinic where it's like you're going to have somebody who's qualified sign off on the final schedule, but maybe it's able to find disjunctions or look for other patterns that you're not considering and give you some additional options there underneath the three that you selected. So it's like it's coming right into the workflow. Same model of the world, I think, is kind of the fundamental conceit.
- [41:42]
Sean Falconer
Who's the sort of like, target user of this? And who do you see as like, competitors as essentially.
- [41:47]
Chris Jagannathan
Yeah, this has always been, I think, a tricky question to answer. I think on the user side, because it is cross Persona, right? So it's like we always think about the operational user backwards. And I think even with our smaller customers, we think that way too. So who is the engineer, who is the analyst? Who is the person that, like the kind of metaphor we use is like, that we're building the Iron man or Iron Woman suit for, right? And it's like their life gets better because they can make better decisions through some sort of application or front end. Now, behind that there usually sits a set of Personas, right? There is the data engineer, there is the maybe traditional application engineer. Now there is the data scientist. And kind of it's like a tiger team of folks that are typically working in a circuit with the operational user. But I think. And there are teams that, by the way, just like do dev work and don't have as much operational work. But we always think about, like, if you ask any, like, classic Palantir deployment, like, who were those champions at bp, at, you know, United Airlines, at, you know, pge, at others, it's always like the electrical engineers, the network operations folks, somebody, right? And so I think those are the Users that we orient around and then we have technical Personas in service of. And I think that's probably a different orientation than most software systems that are out there. That's one thing. I think the result of that is because it kind of has this kind of like phylogenic kind of tree structure to the Personas. It means that like the competition thing gets a little confusing too. And so like there's like a cheeky answer and a non cheeky answer that typically give to this. The cheeky answer is like, it's the internal bill, right? It's like when you're done with like the data lake and the analytics thing and you're like, now you need to go into application space. It's like that's kind of the wild west. I gotta like go hire some consultancy or like staff up a dev team and try to build something custom. And like, maybe we have talented people and we do that. But then it's like, oh, the next app, the next app, the next app. And it's like, this is getting crazy. Some people can do it. But I think what we typically get, what we typically hear is like people told us we would be the next Netflix when it came to building internal apps. And it turns out it's hard to do that without like a system that handles all the things that they have built internally. So it's kind of like the internal shop that's trying to do these things though more and more when they see this as a, it's not a buy versus build, as we say. It's like, I buy this so I can build. Especially now with the AI stuff adding more requirements, that's the first thing. The second thing, and I think especially with offerings like our new kind of warp speed offering, which we think about as like an operational kind of manufacturing heavy version of Foundry. And AIP is like, it's the traditional operation software providers. So think about like the world that's locked into ERPs and PLM systems and kind of all of these things that kind of run the core functions of the business. And right now we kind of treat these things as inputs. But more and more we see with large manufacturers, energy companies, utilities, hospital systems, et cetera. It's like, it's great that those systems have the ledgers in them, but it's like I need to model them into my world, my ontology, and then build and be able to permute that, as Jeg said, as my world changes. And so I think there's kind of like legacy operation software and a lot of that's domain specific that I think we end up being much more competitive with than like data and analytics software, which you're usually plugging in.
- [44:37]
Sean Falconer
Okay, you know, Jag, you talked a little bit about this earlier where, you know, when you start to, you know, plug in these units of LLMs, you're dealing with like a non deterministic world. And I think in some fashion, like building with LLMs requires a bit of a different mindset in traditional engineering, where you're kind of used to like sort of algorithmic inputs and outputs, and what you get out is presumably what you expect if you program the algorithm correctly. So. So as Palantir was starting to build in some of these technologies, did it take effort to get the staff essentially comfortable with the idea of building on LLMs and be ready for dealing with these non deterministic boxes?
- [45:15]
Akshay Krishnaswamy
I think it's an interesting take of exactly whether we were comfortable with non determinism. I think no one is comfortable and ready to accept it. I think the example you often see is if you give an LLM, a calculator at all, then suddenly it can do math. Math. I think this is the kind of journey that we've seen, which was the more of that determinism that we can inherently bring to the table, such that the language model is just solving the exact thing that we need solved that we can trust it to do, whether it's summarization or dealing with specific semantic layers, then the less this ends up being an actual conversation of like, hey, are we okay with the risks that we're taking? We explicitly minimize what is a risky interface, what is a thing that is an explicit decision by, like Akshay said, taking the actions that we've already defined as safe paths for a human to take that are already validated, maybe dry run. And then on top we add concepts like evals, evaluations, test suites, where the things where if we're using a language model and we expect the decision to be deterministic, then we should have deterministic tests in the environment. We basically just ended up building the suite of things that would make us confident that we weren't taking risks, that we knew exactly what the language model is doing is the thing that it is adding on top. I almost would argue it's like the first time you're introduced to a reverse proxy. It's just another piece of software engineering tooling that's in your back pocket. It's not something you'd slap everywhere in your system because it makes no sense. You go like Cool. Where are the LLM shaped holes in my organization where a workflow could be better, but I'm not going to turn up to the table and be like cool, well this whole workflow will just put an LLM in front of the APIs and hope for the best.
- [47:01]
Sean Falconer
From an engineering and technology perspective, what do you feel the most excited about with essentially this transformation that Palantir is making now around helping people build these AI powered applications through this platform?
- [47:13]
Chris Jagannathan
I think I almost think about. It's almost like the Venom symbiote from Spider man in my mind is maybe the. It's like it kind of affects every part of the system, right. If you think about more and more detail being built into the ontology system that represents the reality of the operations of the enterprise, I think there's more and more space for the LLMs to be able to interact and add value in the ways that Jag mentioned where it's like, it's like what we're seeing right even right now. It's like the initial versions of how we incorporated LLMs into data pipelining I'd say was like kind of okay. Then we built some like additional capabilities that were much more targeted and much better received and I think like that's been the case for every part of the elephant. So it's like well how do you then think about to your point Sean, like structuring default ontologies. Turns out you can actually do some more interesting targeted stuff with LLMs if you have that as part of the bootstrapping process with how you define compute and kind of the action framework as well the application building. How can I start to like we've built a lot of the application front ends in the platform. You can also build using your own REACT frameworks and environments and other things using the SDK. But even the way we build applications in the system is still very pre LLM. So it's like why is this not more interactive and helping me kind of like contour my way through. That's the name of a product, pun intended. Sorry. It's of just like being able to get to an application state that makes sense as an non technical user and then work with it. Like we're like right now injecting the LLMs into the existing frameworks. I'd say still enlarged and it's like. But there's going to be entirely new experiences that can be imagined in the application space and I'm particularly excited by just seeing people push the frontier and how much productive like the reverse proxy analogy I love, like, how many critical junction points can you actually bring this to that are value additive? And what can we learn about, like, how to then build more and more capability around those points? And I think, like, this is where, like, being forward deployed and having all of these deep touch points with all of our customers is really, I think, showing us where the wheat is from the chaff and helping us continuously orient and call BS on even just internal assumptions we have. Right.
- [49:06]
Akshay Krishnaswamy
My take is almost like very low level in comparison, which is like, for the individual developer, I think we're in a very exciting period of new technology. Comes up on a Monday, you're like, I want to try it. And as an individual who spends their weekends tinkering on software projects, I'll spend more time trying to set up whatever this new thing is working out, if there's a dockerized version of it, or whether the platform or my laptop will actually even run this model. And then it's a whole different story about whether I can actually use it in a meaningful workflow and the ability to come up with these new technologies that are being delivered to us daily at the moment and then bring them into a workflow that we would consider productionized and say, like, cool, I'm actually going to bring this into an AI infrastructure system such that I can use it against my ontology, against actions that I can actually define. A workflow that will use this video model, for example, makes a huge difference in terms of me going from 0 to 1 in terms of actually building something that's usable. Because I think at the start when we didn't have this tooling, I actually now use Palantir for my own home projects. I would spend so much time just like, spinning up some new service and just trying to get the infrastructure right just so I can start making the application on top and being able to go like, cool, this is the thing that's different. And then this is actually everything that I already had in my system that I can now wire into this container. Whatever makes a huge difference to the velocity at which you can start adopting these new technologies.
- [50:39]
Sean Falconer
Yeah, absolutely. We're coming up on time. So, Akshay and Jag, I want to thank you so much for being here. This was a lot of fun.
- [50:45]
Chris Jagannathan
Thanks so much, Sean.
- [50:46]
Akshay Krishnaswamy
Yeah, thanks for having us.
- [50:47]
Chris Jagannathan
This is awesome.
- [50:48]
Akshay Krishnaswamy
Cheers.